Oct 30 21:36:54.529784 ip-10-0-139-46 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 21:36:54.976957 ip-10-0-139-46 kubenswrapper[2568]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:36:54.976957 ip-10-0-139-46 kubenswrapper[2568]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 21:36:54.976957 ip-10-0-139-46 kubenswrapper[2568]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:36:54.976957 ip-10-0-139-46 kubenswrapper[2568]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:36:54.984185 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984082    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:36:54.984185 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984183    2568 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:36:54.984185 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984187    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:36:54.984185 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984190    2568 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:36:54.984185 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984193    2568 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:36:54.984345 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984197    2568 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:36:54.984345 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984200    2568 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:36:54.984345 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984202    2568 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:36:54.984345 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984205    2568 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:36:54.984345 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984207    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:36:54.984345 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984210    2568 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:36:54.984345 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984213    2568 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:36:54.984345 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984215    2568 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:36:54.984345 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984218    2568 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:36:54.984345 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984221    2568 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:36:54.984345 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984224    2568 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:36:54.984345 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984226    2568 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:36:54.984345 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984229    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:36:54.984345 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984231    2568 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:36:54.984345 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984234    2568 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:36:54.984345 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984236    2568 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:36:54.984345 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984239    2568 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:36:54.984345 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984259    2568 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:36:54.984345 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984262    2568 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:36:54.984801 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984265    2568 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:36:54.984801 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984275    2568 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:36:54.984801 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984281    2568 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:36:54.984801 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984284    2568 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:36:54.984801 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984287    2568 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:36:54.984801 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984290    2568 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:36:54.984801 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984292    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:36:54.984801 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984295    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:36:54.984801 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984297    2568 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:36:54.984801 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984301    2568 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:36:54.984801 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984303    2568 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:36:54.984801 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984306    2568 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:36:54.984801 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984309    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:36:54.984801 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984312    2568 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:36:54.984801 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984315    2568 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:36:54.984801 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984318    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:36:54.984801 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984321    2568 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:36:54.984801 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984325    2568 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:36:54.984801 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984328    2568 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:36:54.984801 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984330    2568 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:36:54.985273 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984333    2568 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:36:54.985273 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984336    2568 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:36:54.985273 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984339    2568 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:36:54.985273 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984342    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:36:54.985273 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984345    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:36:54.985273 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984348    2568 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:36:54.985273 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984351    2568 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:36:54.985273 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984353    2568 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:36:54.985273 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984356    2568 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:36:54.985273 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984359    2568 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:36:54.985273 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984363    2568 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:36:54.985273 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984365    2568 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:36:54.985273 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984368    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:36:54.985273 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984371    2568 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:36:54.985273 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984374    2568 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:36:54.985273 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984377    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:36:54.985273 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984380    2568 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:36:54.985273 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984382    2568 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:36:54.985273 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984385    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:36:54.985273 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984388    2568 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:36:54.985769 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984391    2568 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:36:54.985769 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984393    2568 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:36:54.985769 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984396    2568 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:36:54.985769 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984399    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:36:54.985769 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984402    2568 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:36:54.985769 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984405    2568 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:36:54.985769 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984409    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:36:54.985769 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984411    2568 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:36:54.985769 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984414    2568 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:36:54.985769 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984417    2568 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:36:54.985769 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984420    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:36:54.985769 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984422    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:36:54.985769 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984437    2568 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:36:54.985769 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984440    2568 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:36:54.985769 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984442    2568 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:36:54.985769 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984445    2568 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:36:54.985769 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984447    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:36:54.985769 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984450    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:36:54.985769 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984453    2568 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:36:54.985769 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984455    2568 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:36:54.986264 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984458    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:36:54.986264 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984460    2568 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:36:54.986264 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984463    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:36:54.986264 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984466    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:36:54.986264 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984469    2568 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:36:54.986264 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984471    2568 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:36:54.986264 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984474    2568 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:36:54.986264 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984476    2568 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:36:54.986264 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984479    2568 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:36:54.986264 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984482    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:36:54.986264 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984484    2568 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:36:54.986264 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984488    2568 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:36:54.986264 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984492    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:36:54.986264 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984495    2568 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:36:54.986264 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984498    2568 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:36:54.986264 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984501    2568 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:36:54.986264 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984504    2568 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:36:54.986264 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984508    2568 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:36:54.986264 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984512    2568 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:36:54.986721 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984514    2568 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:36:54.986721 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984517    2568 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:36:54.986721 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.984519    2568 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:36:54.986721 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985411    2568 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:36:54.986721 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985419    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:36:54.986721 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985422    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:36:54.986721 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985438    2568 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:36:54.986721 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985441    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:36:54.986721 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985444    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:36:54.986721 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985447    2568 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:36:54.986721 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985449    2568 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:36:54.986721 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985453    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:36:54.986721 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985456    2568 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:36:54.986721 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985459    2568 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:36:54.986721 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985462    2568 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:36:54.986721 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985464    2568 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:36:54.986721 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985467    2568 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:36:54.986721 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985471    2568 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:36:54.986721 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985473    2568 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:36:54.986721 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985476    2568 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:36:54.987197 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985479    2568 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:36:54.987197 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985481    2568 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:36:54.987197 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985484    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:36:54.987197 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985487    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:36:54.987197 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985489    2568 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:36:54.987197 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985492    2568 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:36:54.987197 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985495    2568 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:36:54.987197 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985497    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:36:54.987197 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985501    2568 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:36:54.987197 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985503    2568 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:36:54.987197 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985506    2568 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:36:54.987197 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985509    2568 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:36:54.987197 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985512    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:36:54.987197 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985514    2568 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:36:54.987197 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985517    2568 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:36:54.987197 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985520    2568 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:36:54.987197 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985522    2568 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:36:54.987197 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985525    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:36:54.987197 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985527    2568 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:36:54.987670 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985530    2568 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:36:54.987670 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985532    2568 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:36:54.987670 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985535    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:36:54.987670 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985537    2568 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:36:54.987670 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985540    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:36:54.987670 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985542    2568 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:36:54.987670 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985545    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:36:54.987670 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985547    2568 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:36:54.987670 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985550    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:36:54.987670 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985553    2568 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:36:54.987670 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985555    2568 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:36:54.987670 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985558    2568 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:36:54.987670 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985561    2568 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:36:54.987670 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985564    2568 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:36:54.987670 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985566    2568 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:36:54.987670 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985569    2568 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:36:54.987670 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985572    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:36:54.987670 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985574    2568 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:36:54.987670 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985577    2568 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:36:54.987670 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985580    2568 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:36:54.988143 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985582    2568 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:36:54.988143 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985585    2568 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:36:54.988143 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985587    2568 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:36:54.988143 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985590    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:36:54.988143 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985594    2568 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:36:54.988143 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985596    2568 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:36:54.988143 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985599    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:36:54.988143 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985601    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:36:54.988143 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985612    2568 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:36:54.988143 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985617    2568 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:36:54.988143 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985621    2568 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:36:54.988143 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985624    2568 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:36:54.988143 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985626    2568 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:36:54.988143 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985629    2568 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:36:54.988143 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985632    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:36:54.988143 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985634    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:36:54.988143 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985637    2568 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:36:54.988143 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985640    2568 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:36:54.988143 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985643    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:36:54.988609 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985645    2568 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:36:54.988609 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985648    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:36:54.988609 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985651    2568 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:36:54.988609 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985653    2568 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:36:54.988609 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985656    2568 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:36:54.988609 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985660    2568 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:36:54.988609 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985664    2568 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:36:54.988609 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985668    2568 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:36:54.988609 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985670    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:36:54.988609 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985674    2568 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:36:54.988609 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985677    2568 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:36:54.988609 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985679    2568 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:36:54.988609 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985682    2568 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:36:54.988609 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985684    2568 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:36:54.988609 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985687    2568 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:36:54.988609 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985689    2568 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:36:54.988609 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985692    2568 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:36:54.988609 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985695    2568 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:36:54.988609 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985697    2568 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:36:54.988609 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985700    2568 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:36:54.989081 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985702    2568 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:36:54.989081 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985705    2568 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:36:54.989081 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985707    2568 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:36:54.989081 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985710    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:36:54.989081 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985712    2568 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:36:54.989081 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985715    2568 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:36:54.989081 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985717    2568 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:36:54.989081 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985720    2568 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:36:54.989081 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985723    2568 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:36:54.989081 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985726    2568 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:36:54.989081 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.985729    2568 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:36:54.989081 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986267    2568 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:36:54.989081 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986273    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:36:54.989081 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986276    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:36:54.989081 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986280    2568 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:36:54.989081 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986283    2568 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:36:54.989081 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986286    2568 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:36:54.989081 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986290    2568 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:36:54.989081 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986295    2568 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:36:54.989537 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986298    2568 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:36:54.989537 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986300    2568 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:36:54.989537 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986303    2568 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:36:54.989537 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986306    2568 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:36:54.989537 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986309    2568 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:36:54.989537 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986311    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:36:54.989537 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986314    2568 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:36:54.989537 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986317    2568 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:36:54.989537 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986319    2568 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:36:54.989537 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986321    2568 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:36:54.989537 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986325    2568 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:36:54.989537 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986328    2568 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:36:54.989537 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986331    2568 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:36:54.989537 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986333    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:36:54.989537 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986336    2568 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:36:54.989537 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986339    2568 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:36:54.989537 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986342    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:36:54.989537 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986344    2568 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:36:54.989537 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986347    2568 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:36:54.989537 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986349    2568 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:36:54.990012 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986352    2568 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:36:54.990012 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986355    2568 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:36:54.990012 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986358    2568 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:36:54.990012 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986362    2568 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:36:54.990012 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986365    2568 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:36:54.990012 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986368    2568 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:36:54.990012 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986370    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:36:54.990012 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986373    2568 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:36:54.990012 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986376    2568 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:36:54.990012 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986378    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:36:54.990012 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986381    2568 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:36:54.990012 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986384    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:36:54.990012 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986387    2568 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:36:54.990012 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986389    2568 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:36:54.990012 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986392    2568 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:36:54.990012 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986394    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:36:54.990012 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986397    2568 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:36:54.990012 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986399    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:36:54.990012 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986402    2568 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:36:54.990475 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986404    2568 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:36:54.990475 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986409    2568 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:36:54.990475 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986412    2568 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:36:54.990475 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986415    2568 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:36:54.990475 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986417    2568 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:36:54.990475 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986420    2568 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:36:54.990475 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986422    2568 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:36:54.990475 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986437    2568 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:36:54.990475 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986440    2568 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:36:54.990475 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986443    2568 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:36:54.990475 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986446    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:36:54.990475 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986448    2568 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:36:54.990475 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986452    2568 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:36:54.990475 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986455    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:36:54.990475 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986457    2568 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:36:54.990475 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986460    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:36:54.990475 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986462    2568 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:36:54.990475 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986465    2568 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:36:54.990475 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986468    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:36:54.990973 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986471    2568 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:36:54.990973 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986473    2568 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:36:54.990973 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986476    2568 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:36:54.990973 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986478    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:36:54.990973 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986481    2568 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:36:54.990973 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986483    2568 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:36:54.990973 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986487    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:36:54.990973 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986489    2568 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:36:54.990973 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986492    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:36:54.990973 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986494    2568 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:36:54.990973 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986497    2568 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:36:54.990973 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986500    2568 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:36:54.990973 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986502    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:36:54.990973 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986504    2568 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:36:54.990973 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986509    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:36:54.990973 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986511    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:36:54.990973 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986514    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:36:54.990973 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986516    2568 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:36:54.990973 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986519    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:36:54.991415 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986521    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:36:54.991415 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986524    2568 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:36:54.991415 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986526    2568 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:36:54.991415 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986529    2568 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:36:54.991415 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986531    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:36:54.991415 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986534    2568 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:36:54.991415 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986536    2568 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:36:54.991415 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986539    2568 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:36:54.991415 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986542    2568 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:36:54.991415 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986545    2568 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:36:54.991415 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986547    2568 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:36:54.991415 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986551    2568 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:36:54.991415 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986554    2568 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:36:54.991415 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986556    2568 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:36:54.991415 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986559    2568 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:36:54.991415 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986561    2568 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:36:54.991415 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986564    2568 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:36:54.991415 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986566    2568 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:36:54.991415 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986569    2568 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:36:54.991415 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986571    2568 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:36:54.991893 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.986574    2568 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:36:54.991893 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987155    2568 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 21:36:54.991893 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987169    2568 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 21:36:54.991893 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987176    2568 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 21:36:54.991893 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987181    2568 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 21:36:54.991893 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987185    2568 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 21:36:54.991893 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987189    2568 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 21:36:54.991893 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987195    2568 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 21:36:54.991893 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987201    2568 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 21:36:54.991893 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987204    2568 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 21:36:54.991893 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987208    2568 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 21:36:54.991893 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987211    2568 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 21:36:54.991893 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987215    2568 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 21:36:54.991893 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987218    2568 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 21:36:54.991893 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987222    2568 flags.go:64] FLAG: --cgroup-root=""
Oct 30 21:36:54.991893 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987224    2568 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 21:36:54.991893 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987228    2568 flags.go:64] FLAG: --client-ca-file=""
Oct 30 21:36:54.991893 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987231    2568 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 21:36:54.991893 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987234    2568 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 21:36:54.991893 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987241    2568 flags.go:64] FLAG: --cluster-domain=""
Oct 30 21:36:54.991893 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987244    2568 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 21:36:54.991893 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987247    2568 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 21:36:54.991893 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987251    2568 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 21:36:54.991893 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987254    2568 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 21:36:54.992468 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987258    2568 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 21:36:54.992468 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987262    2568 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 21:36:54.992468 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987265    2568 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 21:36:54.992468 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987269    2568 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 21:36:54.992468 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987272    2568 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 21:36:54.992468 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987275    2568 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 21:36:54.992468 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987278    2568 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 21:36:54.992468 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987281    2568 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 21:36:54.992468 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987285    2568 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 21:36:54.992468 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987290    2568 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 21:36:54.992468 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987294    2568 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 21:36:54.992468 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987297    2568 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 21:36:54.992468 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987300    2568 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 21:36:54.992468 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987303    2568 flags.go:64] FLAG: --enable-server="true"
Oct 30 21:36:54.992468 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987306    2568 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 21:36:54.992468 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987311    2568 flags.go:64] FLAG: --event-burst="100"
Oct 30 21:36:54.992468 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987314    2568 flags.go:64] FLAG: --event-qps="50"
Oct 30 21:36:54.992468 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987318    2568 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 21:36:54.992468 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987321    2568 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 21:36:54.992468 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987324    2568 flags.go:64] FLAG: --eviction-hard=""
Oct 30 21:36:54.992468 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987329    2568 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 21:36:54.992468 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987332    2568 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 21:36:54.992468 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987335    2568 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 21:36:54.992468 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987338    2568 flags.go:64] FLAG: --eviction-soft=""
Oct 30 21:36:54.992468 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987342    2568 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 21:36:54.993080 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987345    2568 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 21:36:54.993080 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987348    2568 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 21:36:54.993080 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987352    2568 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 21:36:54.993080 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987356    2568 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 21:36:54.993080 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987359    2568 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 21:36:54.993080 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987362    2568 flags.go:64] FLAG: --feature-gates=""
Oct 30 21:36:54.993080 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987367    2568 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 21:36:54.993080 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987370    2568 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 21:36:54.993080 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987373    2568 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 21:36:54.993080 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987377    2568 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 21:36:54.993080 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987380    2568 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 21:36:54.993080 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987383    2568 flags.go:64] FLAG: --help="false"
Oct 30 21:36:54.993080 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987386    2568 flags.go:64] FLAG: --hostname-override="ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:36:54.993080 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987390    2568 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 21:36:54.993080 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987393    2568 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 21:36:54.993080 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987396    2568 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 21:36:54.993080 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987399    2568 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 21:36:54.993080 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987403    2568 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 21:36:54.993080 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987407    2568 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 21:36:54.993080 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987410    2568 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 21:36:54.993080 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987412    2568 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 21:36:54.993080 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987415    2568 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 21:36:54.993080 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987419    2568 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 21:36:54.993771 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987422    2568 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 21:36:54.993771 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987442    2568 flags.go:64] FLAG: --kube-reserved=""
Oct 30 21:36:54.993771 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987445    2568 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 21:36:54.993771 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987448    2568 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 21:36:54.993771 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987452    2568 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 21:36:54.993771 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987455    2568 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 21:36:54.993771 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987458    2568 flags.go:64] FLAG: --lock-file=""
Oct 30 21:36:54.993771 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987461    2568 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 21:36:54.993771 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987465    2568 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 21:36:54.993771 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987468    2568 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 21:36:54.993771 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987478    2568 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 21:36:54.993771 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987483    2568 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 21:36:54.993771 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987486    2568 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 21:36:54.993771 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987490    2568 flags.go:64] FLAG: --logging-format="text"
Oct 30 21:36:54.993771 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987493    2568 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 21:36:54.993771 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987496    2568 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 21:36:54.993771 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987499    2568 flags.go:64] FLAG: --manifest-url=""
Oct 30 21:36:54.993771 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987502    2568 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 21:36:54.993771 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987507    2568 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 21:36:54.993771 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987510    2568 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 21:36:54.993771 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987514    2568 flags.go:64] FLAG: --max-pods="110"
Oct 30 21:36:54.993771 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987517    2568 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 21:36:54.993771 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987521    2568 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 21:36:54.993771 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987524    2568 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 21:36:54.993771 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987526    2568 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 21:36:54.994469 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987529    2568 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 21:36:54.994469 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987533    2568 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 21:36:54.994469 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987537    2568 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 21:36:54.994469 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987544    2568 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 21:36:54.994469 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987548    2568 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 21:36:54.994469 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987551    2568 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 21:36:54.994469 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987554    2568 flags.go:64] FLAG: --pod-cidr=""
Oct 30 21:36:54.994469 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987557    2568 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 21:36:54.994469 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987560    2568 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 21:36:54.994469 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987563    2568 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 21:36:54.994469 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987566    2568 flags.go:64] FLAG: --port="10250"
Oct 30 21:36:54.994469 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987569    2568 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 21:36:54.994469 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987572    2568 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-0b5ea134fd6aead4a"
Oct 30 21:36:54.994469 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987576    2568 flags.go:64] FLAG: --qos-reserved=""
Oct 30 21:36:54.994469 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987579    2568 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 21:36:54.994469 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987582    2568 flags.go:64] FLAG: --register-node="true"
Oct 30 21:36:54.994469 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987585    2568 flags.go:64] FLAG: --register-with-taints=""
Oct 30 21:36:54.994469 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987589    2568 flags.go:64] FLAG: --registry-burst="10"
Oct 30 21:36:54.994469 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987594    2568 flags.go:64] FLAG: --registry-qps="5"
Oct 30 21:36:54.994469 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987597    2568 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 21:36:54.994469 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987600    2568 flags.go:64] FLAG: --reserved-memory=""
Oct 30 21:36:54.994469 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987604    2568 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 21:36:54.994469 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987608    2568 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 21:36:54.994469 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987611    2568 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 21:36:54.994469 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987615    2568 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 21:36:54.994469 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987618    2568 flags.go:64] FLAG: --runonce="false"
Oct 30 21:36:54.995096 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987621    2568 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 21:36:54.995096 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987624    2568 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 21:36:54.995096 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987627    2568 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 21:36:54.995096 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987630    2568 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 21:36:54.995096 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987633    2568 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 21:36:54.995096 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987637    2568 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 21:36:54.995096 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987640    2568 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 21:36:54.995096 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987643    2568 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 21:36:54.995096 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987646    2568 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 21:36:54.995096 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987650    2568 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 21:36:54.995096 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987654    2568 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 21:36:54.995096 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987657    2568 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 21:36:54.995096 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987660    2568 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 21:36:54.995096 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987663    2568 flags.go:64] FLAG: --system-cgroups=""
Oct 30 21:36:54.995096 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987666    2568 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 21:36:54.995096 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987672    2568 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 21:36:54.995096 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987675    2568 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 21:36:54.995096 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987678    2568 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 21:36:54.995096 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987683    2568 flags.go:64] FLAG: --tls-min-version=""
Oct 30 21:36:54.995096 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987686    2568 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 21:36:54.995096 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987689    2568 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 21:36:54.995096 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987692    2568 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 21:36:54.995096 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987695    2568 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 21:36:54.995096 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987698    2568 flags.go:64] FLAG: --v="2"
Oct 30 21:36:54.995096 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987703    2568 flags.go:64] FLAG: --version="false"
Oct 30 21:36:54.995702 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987708    2568 flags.go:64] FLAG: --vmodule=""
Oct 30 21:36:54.995702 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987712    2568 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 21:36:54.995702 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.987716    2568 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 21:36:54.995702 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.987890    2568 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:36:54.995702 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.987894    2568 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:36:54.995702 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.987897    2568 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:36:54.995702 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.987900    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:36:54.995702 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.987903    2568 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:36:54.995702 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.987906    2568 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:36:54.995702 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.987908    2568 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:36:54.995702 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.987911    2568 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:36:54.995702 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.987914    2568 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:36:54.995702 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.987916    2568 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:36:54.995702 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.987918    2568 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:36:54.995702 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.987921    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:36:54.995702 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.987924    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:36:54.995702 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.987927    2568 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:36:54.995702 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.987930    2568 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:36:54.995702 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.987932    2568 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:36:54.995702 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.987936    2568 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:36:54.996174 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.987939    2568 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:36:54.996174 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.987942    2568 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:36:54.996174 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.987944    2568 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:36:54.996174 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.987947    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:36:54.996174 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.987949    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:36:54.996174 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.987952    2568 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:36:54.996174 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.987955    2568 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:36:54.996174 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.987957    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:36:54.996174 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.987960    2568 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:36:54.996174 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.987962    2568 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:36:54.996174 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.987965    2568 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:36:54.996174 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.987967    2568 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:36:54.996174 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.987971    2568 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:36:54.996174 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.987975    2568 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:36:54.996174 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.987978    2568 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:36:54.996174 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.987982    2568 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:36:54.996174 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.987985    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:36:54.996174 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.987987    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:36:54.996174 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.987990    2568 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:36:54.996174 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.987993    2568 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:36:54.996756 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.987995    2568 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:36:54.996756 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.987998    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:36:54.996756 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988000    2568 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:36:54.996756 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988003    2568 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:36:54.996756 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988005    2568 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:36:54.996756 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988008    2568 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:36:54.996756 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988010    2568 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:36:54.996756 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988013    2568 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:36:54.996756 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988015    2568 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:36:54.996756 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988018    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:36:54.996756 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988021    2568 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:36:54.996756 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988025    2568 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:36:54.996756 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988028    2568 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:36:54.996756 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988030    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:36:54.996756 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988033    2568 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:36:54.996756 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988035    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:36:54.996756 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988037    2568 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:36:54.996756 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988040    2568 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:36:54.996756 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988042    2568 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:36:54.997670 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988045    2568 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:36:54.997670 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988047    2568 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:36:54.997670 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988050    2568 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:36:54.997670 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988052    2568 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:36:54.997670 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988055    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:36:54.997670 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988057    2568 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:36:54.997670 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988060    2568 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:36:54.997670 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988062    2568 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:36:54.997670 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988065    2568 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:36:54.997670 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988068    2568 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:36:54.997670 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988070    2568 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:36:54.997670 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988076    2568 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:36:54.997670 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988079    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:36:54.997670 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988081    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:36:54.997670 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988084    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:36:54.997670 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988087    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:36:54.997670 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988089    2568 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:36:54.997670 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988092    2568 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:36:54.997670 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988094    2568 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:36:54.998197 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988097    2568 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:36:54.998197 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988099    2568 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:36:54.998197 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988102    2568 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:36:54.998197 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988104    2568 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:36:54.998197 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988107    2568 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:36:54.998197 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988112    2568 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:36:54.998197 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988115    2568 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:36:54.998197 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988118    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:36:54.998197 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988121    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:36:54.998197 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988123    2568 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:36:54.998197 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988126    2568 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:36:54.998197 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988129    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:36:54.998197 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988131    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:36:54.998197 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988133    2568 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:36:54.998197 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988136    2568 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:36:54.998197 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988138    2568 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:36:54.998197 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988141    2568 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:36:54.998197 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988144    2568 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:36:54.998197 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988146    2568 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:36:54.998197 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988150    2568 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:36:54.998698 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988154    2568 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:36:54.998698 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988157    2568 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:36:54.998698 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988160    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:36:54.998698 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988162    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:36:54.998698 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988165    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:36:54.998698 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988168    2568 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:36:54.998698 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988170    2568 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:36:54.998698 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988173    2568 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:36:54.998698 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988175    2568 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:36:54.998698 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988178    2568 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:36:54.998698 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988180    2568 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:36:54.998698 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.988207    2568 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:36:54.998698 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988244    2568 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:36:54.998698 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988248    2568 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:36:54.998698 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988251    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:36:54.998698 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988254    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:36:54.999161 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988257    2568 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:36:54.999161 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988260    2568 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:36:54.999161 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988265    2568 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:36:54.999161 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988268    2568 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:36:54.999161 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988271    2568 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:36:54.999161 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988273    2568 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:36:54.999161 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988277    2568 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:36:54.999161 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988281    2568 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:36:54.999161 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988284    2568 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:36:54.999161 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988287    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:36:54.999161 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988290    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:36:54.999161 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988292    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:36:54.999161 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988295    2568 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:36:54.999161 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988298    2568 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:36:54.999161 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988300    2568 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:36:54.999161 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988303    2568 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:36:54.999161 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988305    2568 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:36:54.999161 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988308    2568 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:36:54.999161 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988310    2568 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:36:54.999161 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988313    2568 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:36:54.999674 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988315    2568 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:36:54.999674 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988318    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:36:54.999674 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988321    2568 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:36:54.999674 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988323    2568 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:36:54.999674 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988326    2568 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:36:54.999674 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988328    2568 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:36:54.999674 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988331    2568 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:36:54.999674 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988333    2568 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:36:54.999674 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988336    2568 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:36:54.999674 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988338    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:36:54.999674 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988341    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:36:54.999674 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988343    2568 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:36:54.999674 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988346    2568 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:36:54.999674 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988350    2568 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:36:54.999674 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988354    2568 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:36:54.999674 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988357    2568 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:36:54.999674 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988360    2568 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:36:54.999674 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988362    2568 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:36:54.999674 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988365    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:36:54.999674 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988367    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:36:55.000137 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988370    2568 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:36:55.000137 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988373    2568 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:36:55.000137 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988375    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:36:55.000137 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988378    2568 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:36:55.000137 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988380    2568 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:36:55.000137 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988383    2568 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:36:55.000137 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988385    2568 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:36:55.000137 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988390    2568 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:36:55.000137 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988393    2568 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:36:55.000137 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988397    2568 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:36:55.000137 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988399    2568 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:36:55.000137 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988402    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:36:55.000137 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988404    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:36:55.000137 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988407    2568 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:36:55.000137 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988409    2568 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:36:55.000137 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988412    2568 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:36:55.000137 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988414    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:36:55.000137 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988417    2568 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:36:55.000137 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988420    2568 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:36:55.000639 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988422    2568 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:36:55.000639 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988437    2568 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:36:55.000639 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988439    2568 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:36:55.000639 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988442    2568 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:36:55.000639 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988445    2568 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:36:55.000639 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988447    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:36:55.000639 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988452    2568 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:36:55.000639 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988456    2568 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:36:55.000639 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988459    2568 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:36:55.000639 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988462    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:36:55.000639 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988465    2568 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:36:55.000639 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988467    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:36:55.000639 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988470    2568 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:36:55.000639 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988472    2568 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:36:55.000639 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988476    2568 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:36:55.000639 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988478    2568 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:36:55.000639 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988481    2568 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:36:55.000639 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988483    2568 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:36:55.000639 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988486    2568 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:36:55.000639 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988489    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:36:55.001099 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988491    2568 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:36:55.001099 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988494    2568 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:36:55.001099 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988496    2568 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:36:55.001099 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988499    2568 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:36:55.001099 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988502    2568 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:36:55.001099 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988505    2568 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:36:55.001099 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988507    2568 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:36:55.001099 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988510    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:36:55.001099 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988512    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:36:55.001099 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988515    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:36:55.001099 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988518    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:36:55.001099 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988521    2568 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:36:55.001099 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988523    2568 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:36:55.001099 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988525    2568 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:36:55.001099 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988528    2568 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:36:55.001099 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988530    2568 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:36:55.001099 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988533    2568 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:36:55.001099 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988536    2568 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:36:55.001099 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988540    2568 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:36:55.001557 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988544    2568 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:36:55.001557 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988547    2568 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:36:55.001557 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988550    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:36:55.001557 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.988554    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:36:55.001557 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.996044    2568 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 21:36:55.001557 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.996062    2568 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 21:36:55.001557 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.996079    2568 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 21:36:55.001557 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.996083    2568 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 21:36:55.001557 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996145    2568 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:36:55.001557 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996150    2568 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:36:55.001557 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996154    2568 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:36:55.001557 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996158    2568 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:36:55.001557 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996161    2568 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:36:55.001557 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996163    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:36:55.001557 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996166    2568 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:36:55.001557 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996169    2568 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:36:55.001557 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996171    2568 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:36:55.001557 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996174    2568 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:36:55.001557 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996177    2568 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:36:55.002009 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996180    2568 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:36:55.002009 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996183    2568 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:36:55.002009 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996186    2568 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:36:55.002009 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996189    2568 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:36:55.002009 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996191    2568 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:36:55.002009 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996194    2568 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:36:55.002009 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996197    2568 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:36:55.002009 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996199    2568 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:36:55.002009 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996202    2568 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:36:55.002009 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996205    2568 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:36:55.002009 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996207    2568 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:36:55.002009 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996209    2568 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:36:55.002009 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996212    2568 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:36:55.002009 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996215    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:36:55.002009 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996217    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:36:55.002009 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996220    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:36:55.002009 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996224    2568 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:36:55.002009 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996227    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:36:55.002009 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996230    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:36:55.002467 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996233    2568 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:36:55.002467 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996235    2568 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:36:55.002467 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996238    2568 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:36:55.002467 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996240    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:36:55.002467 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996243    2568 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:36:55.002467 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996245    2568 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:36:55.002467 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996248    2568 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:36:55.002467 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996251    2568 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:36:55.002467 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996253    2568 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:36:55.002467 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996256    2568 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:36:55.002467 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996259    2568 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:36:55.002467 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996262    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:36:55.002467 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996264    2568 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:36:55.002467 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996267    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:36:55.002467 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996270    2568 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:36:55.002467 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996272    2568 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:36:55.002467 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996275    2568 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:36:55.002467 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996277    2568 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:36:55.002467 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996280    2568 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:36:55.002467 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996282    2568 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:36:55.002984 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996285    2568 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:36:55.002984 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996288    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:36:55.002984 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996290    2568 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:36:55.002984 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996293    2568 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:36:55.002984 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996295    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:36:55.002984 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996298    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:36:55.002984 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996301    2568 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:36:55.002984 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996303    2568 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:36:55.002984 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996306    2568 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:36:55.002984 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996309    2568 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:36:55.002984 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996312    2568 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:36:55.002984 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996315    2568 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:36:55.002984 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996318    2568 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:36:55.002984 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996321    2568 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:36:55.002984 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996323    2568 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:36:55.002984 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996326    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:36:55.002984 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996329    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:36:55.002984 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996331    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:36:55.002984 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996334    2568 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:36:55.003417 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996337    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:36:55.003417 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996340    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:36:55.003417 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996342    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:36:55.003417 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996345    2568 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:36:55.003417 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996347    2568 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:36:55.003417 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996350    2568 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:36:55.003417 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996353    2568 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:36:55.003417 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996355    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:36:55.003417 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996358    2568 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:36:55.003417 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996360    2568 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:36:55.003417 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996363    2568 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:36:55.003417 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996366    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:36:55.003417 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996369    2568 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:36:55.003417 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996371    2568 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:36:55.003417 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996374    2568 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:36:55.003417 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996377    2568 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:36:55.003417 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996379    2568 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:36:55.003417 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996382    2568 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:36:55.003417 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996385    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:36:55.003417 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996388    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:36:55.004010 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996390    2568 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:36:55.004010 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996392    2568 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:36:55.004010 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996395    2568 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:36:55.004010 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996397    2568 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:36:55.004010 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996401    2568 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:36:55.004010 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996405    2568 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:36:55.004010 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996409    2568 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:36:55.004010 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996413    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:36:55.004010 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996416    2568 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:36:55.004010 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996419    2568 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:36:55.004010 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996423    2568 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:36:55.004010 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996445    2568 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:36:55.004010 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996448    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:36:55.004010 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996451    2568 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:36:55.004010 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996453    2568 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:36:55.004010 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996456    2568 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:36:55.004010 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.996459    2568 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:36:55.004399 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.996491    2568 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:36:55.004399 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999367    2568 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:36:55.004399 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999384    2568 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:36:55.004399 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999388    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:36:55.004399 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999392    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:36:55.004399 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999395    2568 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:36:55.004399 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999398    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:36:55.004399 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999401    2568 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:36:55.004399 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999404    2568 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:36:55.004399 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999407    2568 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:36:55.004399 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999410    2568 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:36:55.004399 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999412    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:36:55.004399 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999415    2568 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:36:55.004399 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999418    2568 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:36:55.004399 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999421    2568 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:36:55.004399 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999423    2568 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:36:55.004792 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999442    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:36:55.004792 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999445    2568 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:36:55.004792 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999448    2568 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:36:55.004792 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999451    2568 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:36:55.004792 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999453    2568 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:36:55.004792 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999456    2568 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:36:55.004792 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999459    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:36:55.004792 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999461    2568 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:36:55.004792 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999464    2568 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:36:55.004792 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999471    2568 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:36:55.004792 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999474    2568 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:36:55.004792 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999477    2568 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:36:55.004792 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999480    2568 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:36:55.004792 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999483    2568 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:36:55.004792 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999485    2568 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:36:55.004792 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999488    2568 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:36:55.004792 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999490    2568 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:36:55.004792 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999493    2568 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:36:55.004792 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999496    2568 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:36:55.004792 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999500    2568 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:36:55.005259 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999503    2568 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:36:55.005259 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999505    2568 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:36:55.005259 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999508    2568 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:36:55.005259 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999511    2568 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:36:55.005259 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999514    2568 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:36:55.005259 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999517    2568 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:36:55.005259 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999519    2568 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:36:55.005259 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999522    2568 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:36:55.005259 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999526    2568 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:36:55.005259 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999531    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:36:55.005259 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999534    2568 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:36:55.005259 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999537    2568 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:36:55.005259 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999541    2568 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:36:55.005259 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999544    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:36:55.005259 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999547    2568 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:36:55.005259 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999550    2568 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:36:55.005259 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999553    2568 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:36:55.005259 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999556    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:36:55.005259 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999558    2568 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:36:55.005725 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999561    2568 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:36:55.005725 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999565    2568 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:36:55.005725 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999569    2568 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:36:55.005725 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999572    2568 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:36:55.005725 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999575    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:36:55.005725 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999578    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:36:55.005725 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999580    2568 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:36:55.005725 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999583    2568 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:36:55.005725 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999585    2568 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:36:55.005725 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999588    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:36:55.005725 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999591    2568 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:36:55.005725 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999595    2568 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:36:55.005725 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999597    2568 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:36:55.005725 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999600    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:36:55.005725 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999603    2568 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:36:55.005725 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999606    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:36:55.005725 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999608    2568 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:36:55.005725 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999611    2568 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:36:55.005725 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999614    2568 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:36:55.006183 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999616    2568 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:36:55.006183 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999619    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:36:55.006183 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999622    2568 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:36:55.006183 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999624    2568 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:36:55.006183 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999627    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:36:55.006183 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999629    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:36:55.006183 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999632    2568 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:36:55.006183 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999634    2568 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:36:55.006183 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999637    2568 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:36:55.006183 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999639    2568 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:36:55.006183 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999643    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:36:55.006183 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999646    2568 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:36:55.006183 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999649    2568 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:36:55.006183 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999651    2568 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:36:55.006183 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999654    2568 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:36:55.006183 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999656    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:36:55.006183 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999659    2568 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:36:55.006183 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999662    2568 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:36:55.006183 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999664    2568 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:36:55.006183 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999667    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:36:55.006719 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999669    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:36:55.006719 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999672    2568 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:36:55.006719 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999675    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:36:55.006719 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999678    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:36:55.006719 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999681    2568 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:36:55.006719 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999684    2568 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:36:55.006719 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999686    2568 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:36:55.006719 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999689    2568 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:36:55.006719 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999692    2568 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:36:55.006719 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999695    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:36:55.006719 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999697    2568 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:36:55.006719 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999700    2568 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:36:55.006719 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999702    2568 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:36:55.006719 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:54.999737    2568 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:36:55.006719 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999778    2568 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:36:55.006719 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999782    2568 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:36:55.007116 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999784    2568 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:36:55.007116 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999787    2568 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:36:55.007116 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999790    2568 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:36:55.007116 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999793    2568 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:36:55.007116 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999795    2568 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:36:55.007116 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999798    2568 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:36:55.007116 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999801    2568 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:36:55.007116 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999805    2568 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:36:55.007116 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999808    2568 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:36:55.007116 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999811    2568 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:36:55.007116 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999813    2568 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:36:55.007116 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999816    2568 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:36:55.007116 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999819    2568 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:36:55.007116 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999821    2568 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:36:55.007116 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999824    2568 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:36:55.007116 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999828    2568 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:36:55.007116 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999831    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:36:55.007116 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999834    2568 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:36:55.007116 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999837    2568 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:36:55.007582 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999839    2568 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:36:55.007582 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999842    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:36:55.007582 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999844    2568 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:36:55.007582 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999847    2568 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:36:55.007582 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999850    2568 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:36:55.007582 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999852    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:36:55.007582 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999855    2568 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:36:55.007582 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999858    2568 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:36:55.007582 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999862    2568 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:36:55.007582 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999865    2568 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:36:55.007582 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999868    2568 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:36:55.007582 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999871    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:36:55.007582 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999873    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:36:55.007582 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999876    2568 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:36:55.007582 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999878    2568 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:36:55.007582 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999881    2568 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:36:55.007582 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999883    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:36:55.007582 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999886    2568 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:36:55.007582 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999889    2568 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:36:55.008026 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999892    2568 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:36:55.008026 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999894    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:36:55.008026 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999897    2568 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:36:55.008026 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999900    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:36:55.008026 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999902    2568 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:36:55.008026 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999905    2568 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:36:55.008026 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999908    2568 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:36:55.008026 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999910    2568 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:36:55.008026 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999913    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:36:55.008026 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999915    2568 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:36:55.008026 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999917    2568 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:36:55.008026 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999920    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:36:55.008026 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999923    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:36:55.008026 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999926    2568 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:36:55.008026 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999928    2568 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:36:55.008026 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999931    2568 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:36:55.008026 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999933    2568 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:36:55.008026 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999936    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:36:55.008026 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999938    2568 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:36:55.008026 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999941    2568 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:36:55.008551 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999944    2568 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:36:55.008551 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999947    2568 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:36:55.008551 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999949    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:36:55.008551 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999952    2568 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:36:55.008551 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999955    2568 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:36:55.008551 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999957    2568 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:36:55.008551 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999960    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:36:55.008551 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999962    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:36:55.008551 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999965    2568 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:36:55.008551 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999967    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:36:55.008551 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999970    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:36:55.008551 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999975    2568 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:36:55.008551 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999977    2568 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:36:55.008551 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999980    2568 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:36:55.008551 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999983    2568 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:36:55.008551 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999986    2568 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:36:55.008551 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999989    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:36:55.008551 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999992    2568 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:36:55.008551 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999994    2568 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:36:55.008551 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:54.999997    2568 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:36:55.009008 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:55.000000    2568 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:36:55.009008 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:55.000002    2568 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:36:55.009008 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:55.000005    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:36:55.009008 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:55.000007    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:36:55.009008 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:55.000010    2568 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:36:55.009008 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:55.000013    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:36:55.009008 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:55.000016    2568 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:36:55.009008 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:55.000018    2568 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:36:55.009008 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:55.000021    2568 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:36:55.009008 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:55.000024    2568 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:36:55.009008 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:55.000026    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:36:55.009008 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:55.000029    2568 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:36:55.009008 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:55.000031    2568 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:36:55.009008 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:55.000034    2568 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:36:55.009008 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:55.000037    2568 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:36:55.009008 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:55.000039    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:36:55.009008 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:55.000042    2568 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:36:55.009008 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:55.000044    2568 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:36:55.009008 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:55.000047    2568 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:36:55.009008 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:55.000050    2568 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:36:55.009487 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:55.000052    2568 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:36:55.009487 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:55.000055    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:36:55.009487 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:55.000058    2568 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:36:55.009487 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:55.000063    2568 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:36:55.009487 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:55.000066    2568 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:36:55.009487 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:55.000069    2568 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:36:55.009487 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.000131    2568 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 21:36:55.009487 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.002393    2568 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 30 21:36:55.009487 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.003503    2568 server.go:1014] "Starting client certificate rotation"
Oct 30 21:36:55.009487 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.003596    2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 21:36:55.009487 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.003631    2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 21:36:55.009487 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.006056    2568 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:36:55.009487 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.007404    2568 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:36:55.017994 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.017785    2568 log.go:25] "Validated CRI v1 runtime API"
Oct 30 21:36:55.021462 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.021444    2568 log.go:25] "Validated CRI v1 image API"
Oct 30 21:36:55.022387 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.022359    2568 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 21:36:55.030246 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.030223    2568 fs.go:135] Filesystem UUIDs: map[6ac87993-cbcf-4e29-8e44-b6d395f7dc1e:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 d3d60f1a-6035-4228-8334-dbfe0aa51694:/dev/nvme0n1p4]
Oct 30 21:36:55.030328 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.030246    2568 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 30 21:36:55.036317 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.036185    2568 manager.go:217] Machine: {Timestamp:2026-10-30 21:36:55.034860673 +0000 UTC m=+0.377162138 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3005941 MemoryCapacity:33164341248 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a6fbaec3b62344ab46bc006672533 SystemUUID:ec2a6fba-ec3b-6234-4ab4-6bc006672533 BootID:661fa28e-b79d-4a20-b743-5bee4b6c0176 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582172672 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:06:ff:fc:54:30:7f Speed:0 Mtu:9001} {Name:ens5 MacAddress:06:ff:fc:54:30:7f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6a:15:04:48:34:95 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164341248 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 21:36:55.036317 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.036316    2568 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 21:36:55.036420 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.036406    2568 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 21:36:55.039267 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.039236    2568 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 21:36:55.039411 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.039270    2568 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-46.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 21:36:55.039473 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.039413    2568 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 21:36:55.039473 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.039421    2568 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 21:36:55.039473 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.039469    2568 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:36:55.040330 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.040320    2568 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:36:55.041115 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.041104    2568 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 21:36:55.045163 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.045150    2568 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 21:36:55.045291 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.045281    2568 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 21:36:55.046028 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.046019    2568 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 21:36:55.046070 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.046033    2568 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 21:36:55.046070 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.046047    2568 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 21:36:55.046070 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.046059    2568 kubelet.go:405] "Adding apiserver pod source"
Oct 30 21:36:55.046070 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.046068    2568 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 21:36:55.049125 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.049101    2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:36:55.052008 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.051987    2568 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 21:36:55.056671 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.056654    2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:36:55.057570 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.057557    2568 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 21:36:55.057618 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.057589    2568 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 21:36:55.057773 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.057762    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 21:36:55.057823 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.057779    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 21:36:55.057823 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.057786    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 21:36:55.057823 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.057792    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 21:36:55.057823 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.057798    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 21:36:55.057823 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.057804    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 21:36:55.058535 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.058526    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 21:36:55.058565 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.058536    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 21:36:55.058565 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.058543    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 21:36:55.058565 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.058550    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 21:36:55.058565 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.058559    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 21:36:55.058722 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.058567    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 21:36:55.059823 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.059813    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 21:36:55.059857 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.059823    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 21:36:55.064979 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.064966    2568 server.go:1263] "Started kubelet"
Oct 30 21:36:55.065133 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.065086    2568 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 21:36:55.065350 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.065085    2568 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 21:36:55.065455 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.065377    2568 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 21:36:55.065975 ip-10-0-139-46 systemd[1]: Started Kubernetes Kubelet.
Oct 30 21:36:55.066546 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.066530    2568 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 21:36:55.067923 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.067909    2568 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 21:36:55.070343 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:55.070324    2568 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 30 21:36:55.072934 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.072915    2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 21:36:55.074010 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.073995    2568 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 21:36:55.074117 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:55.074093    2568 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-46.us-west-2.compute.internal\" not found"
Oct 30 21:36:55.074585 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.074565    2568 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 21:36:55.074585 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.074567    2568 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 21:36:55.074713 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.074602    2568 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 21:36:55.074713 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.074701    2568 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 21:36:55.074713 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.074712    2568 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 21:36:55.075388 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.075370    2568 factory.go:55] Registering systemd factory
Oct 30 21:36:55.075443 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.075393    2568 factory.go:223] Registration of the systemd container factory successfully
Oct 30 21:36:55.075646 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.075633    2568 factory.go:153] Registering CRI-O factory
Oct 30 21:36:55.075717 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.075648    2568 factory.go:223] Registration of the crio container factory successfully
Oct 30 21:36:55.075717 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.075707    2568 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 21:36:55.075826 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.075725    2568 factory.go:103] Registering Raw factory
Oct 30 21:36:55.075826 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.075740    2568 manager.go:1199] Started watching for new ooms in manager
Oct 30 21:36:55.076657 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.076642    2568 manager.go:319] Starting recovery of all containers
Oct 30 21:36:55.087504 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.087483    2568 manager.go:324] Recovery completed
Oct 30 21:36:55.088572 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:55.088549    2568 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Oct 30 21:36:55.091576 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.091481    2568 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:36:55.093379 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.093354    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:36:55.093489 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.093392    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:36:55.093489 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.093403    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:36:55.094033 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.094014    2568 cpu_manager.go:226] "Starting" policy="none"
Oct 30 21:36:55.094033 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.094024    2568 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:36:55.094033 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.094028    2568 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 21:36:55.094191 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.094046    2568 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 21:36:55.094731 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.094714    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:36:55.094792 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.094743    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:36:55.094792 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.094756    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:36:55.096213 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.096202    2568 policy_none.go:50] "Start"
Oct 30 21:36:55.096280 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.096217    2568 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 21:36:55.096280 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.096228    2568 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 21:36:55.098397 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.098386    2568 policy_none.go:44] "Start"
Oct 30 21:36:55.148335 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.148287    2568 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 21:36:55.152203 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.152006    2568 manager.go:341] "Starting Device Plugin manager"
Oct 30 21:36:55.152315 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:55.152235    2568 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 21:36:55.152315 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.152246    2568 server.go:87] "Starting device plugin registration server"
Oct 30 21:36:55.152509 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.152497    2568 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 21:36:55.152560 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.152511    2568 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 21:36:55.153059 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.152768    2568 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 21:36:55.153059 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.152849    2568 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 21:36:55.153059 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.152855    2568 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 21:36:55.153441 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:55.153399    2568 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 21:36:55.153674 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:55.153471    2568 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-46.us-west-2.compute.internal\" not found"
Oct 30 21:36:55.198982 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.198944    2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 21:36:55.201004 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.200075    2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 21:36:55.201004 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.200103    2568 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 21:36:55.201004 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.200132    2568 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 21:36:55.201004 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:55.200176    2568 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 30 21:36:55.202738 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.202715    2568 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-46.us-west-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 21:36:55.202842 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:55.202814    2568 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-46.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 30 21:36:55.204063 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:55.202712    2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-46.us-west-2.compute.internal.18e36c5b1479c32c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-46.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-46.us-west-2.compute.internal,},FirstTimestamp:2026-10-30 21:36:55.06493726 +0000 UTC m=+0.407238726,LastTimestamp:2026-10-30 21:36:55.06493726 +0000 UTC m=+0.407238726,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-46.us-west-2.compute.internal,}"
Oct 30 21:36:55.253052 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.252947    2568 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:36:55.254015 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.253996    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:36:55.254113 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.254027    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:36:55.254113 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.254037    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:36:55.254113 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.254064    2568 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:36:55.261074 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:55.261043    2568 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Oct 30 21:36:55.261230 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:55.261077    2568 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Oct 30 21:36:55.261230 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:55.261010    2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-46.us-west-2.compute.internal.18e36c5b162bbd60  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-46.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-46.us-west-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-46.us-west-2.compute.internal,},FirstTimestamp:2026-10-30 21:36:55.0933784 +0000 UTC m=+0.435679866,LastTimestamp:2026-10-30 21:36:55.0933784 +0000 UTC m=+0.435679866,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-46.us-west-2.compute.internal,}"
Oct 30 21:36:55.261362 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:55.261268    2568 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-46.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Oct 30 21:36:55.299932 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.299897    2568 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 21:36:55.300070 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.299964    2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ppvv7"
Oct 30 21:36:55.300872 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.300851    2568 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-46.us-west-2.compute.internal"]
Oct 30 21:36:55.300940 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.300912    2568 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:36:55.301950 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.301932    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:36:55.302046 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.301961    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:36:55.302046 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.301971    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:36:55.304866 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.304854    2568 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:36:55.305054 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.305036    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:36:55.305102 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.305077    2568 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:36:55.305725 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.305708    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:36:55.305808 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.305736    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:36:55.305808 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.305746    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:36:55.305808 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.305759    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:36:55.305808 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.305779    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:36:55.305808 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.305790    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:36:55.307742 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.307727    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:36:55.307809 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.307753    2568 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:36:55.308500 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.308486    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:36:55.308557 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.308510    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:36:55.308557 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.308520    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:36:55.326896 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:55.326869    2568 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-46.us-west-2.compute.internal\" not found" node="ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:36:55.329623 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:55.329592    2568 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-46.us-west-2.compute.internal\" not found" node="ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:36:55.354105 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.354078    2568 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:36:55.360550 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.360529    2568 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:36:55.360623 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:55.360557    2568 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-46.us-west-2.compute.internal\": node \"ip-10-0-139-46.us-west-2.compute.internal\" not found"
Oct 30 21:36:55.378107 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.378069    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/85a4f26b44393e519ecc5f1c6d0f8225-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal\" (UID: \"85a4f26b44393e519ecc5f1c6d0f8225\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:36:55.378107 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.378109    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/85a4f26b44393e519ecc5f1c6d0f8225-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal\" (UID: \"85a4f26b44393e519ecc5f1c6d0f8225\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:36:55.378292 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:55.378135    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1d836aa5280c51c365d2d598465f9d5c-config\") pod \"kube-apiserver-proxy-ip-10-0-139-46.us-west-2.compute.internal\" (UID: \"1d836aa5280c51c365d2d598465f9d5c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:36:56.492073 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:56.492043    2568 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-46.us-west-2.compute.internal\" not found"
Oct 30 21:36:56.494090 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:56.494049    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/85a4f26b44393e519ecc5f1c6d0f8225-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal\" (UID: \"85a4f26b44393e519ecc5f1c6d0f8225\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:36:56.494090 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:56.494091    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1d836aa5280c51c365d2d598465f9d5c-config\") pod \"kube-apiserver-proxy-ip-10-0-139-46.us-west-2.compute.internal\" (UID: \"1d836aa5280c51c365d2d598465f9d5c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:36:56.494266 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:56.494136    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/85a4f26b44393e519ecc5f1c6d0f8225-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal\" (UID: \"85a4f26b44393e519ecc5f1c6d0f8225\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:36:56.494266 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:56.494170    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/85a4f26b44393e519ecc5f1c6d0f8225-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal\" (UID: \"85a4f26b44393e519ecc5f1c6d0f8225\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:36:56.494266 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:56.494170    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/85a4f26b44393e519ecc5f1c6d0f8225-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal\" (UID: \"85a4f26b44393e519ecc5f1c6d0f8225\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:36:56.494266 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:56.494195    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1d836aa5280c51c365d2d598465f9d5c-config\") pod \"kube-apiserver-proxy-ip-10-0-139-46.us-west-2.compute.internal\" (UID: \"1d836aa5280c51c365d2d598465f9d5c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:36:56.592347 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:56.592264    2568 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-46.us-west-2.compute.internal\" not found"
Oct 30 21:36:56.645678 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:56.645650    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:36:56.648770 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:56.648754    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:36:56.692407 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:56.692355    2568 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-46.us-west-2.compute.internal\" not found"
Oct 30 21:36:56.792851 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:56.792818    2568 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-46.us-west-2.compute.internal\" not found"
Oct 30 21:36:56.893991 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:56.893913    2568 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-46.us-west-2.compute.internal\" not found"
Oct 30 21:36:56.994147 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:56.994111    2568 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-46.us-west-2.compute.internal\" not found"
Oct 30 21:36:57.018878 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:57.018857    2568 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 30 21:36:57.018991 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:57.018973    2568 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 21:36:57.019118 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:57.019034    2568 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://ac60eab6f532d4004a67394301a17821-2cf05b1ccc71f3c2.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-139-46.us-west-2.compute.internal.18e36c5b162c2e8e\": read tcp 10.0.139.46:42664->3.151.38.176:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-139-46.us-west-2.compute.internal.18e36c5b162c2e8e  default   5086 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-46.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-139-46.us-west-2.compute.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-139-46.us-west-2.compute.internal,},FirstTimestamp:2026-10-30 21:36:55 +0000 UTC,LastTimestamp:2026-10-30 21:36:55.301975479 +0000 UTC m=+0.644276944,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-46.us-west-2.compute.internal,}"
Oct 30 21:36:57.094787 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:57.094749    2568 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-46.us-west-2.compute.internal\" not found"
Oct 30 21:36:57.195432 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:57.195362    2568 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-46.us-west-2.compute.internal\" not found"
Oct 30 21:36:57.295587 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:57.295545    2568 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-46.us-west-2.compute.internal\" not found"
Oct 30 21:36:57.320802 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:57.320769    2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 21:31:56 +0000 UTC" deadline="2028-08-08 23:56:24.226526098 +0000 UTC"
Oct 30 21:36:57.320802 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:57.320796    2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15554h19m26.905732903s"
Oct 30 21:36:57.366938 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:57.366902    2568 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:36:57.368000 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:57.367983    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:36:57.368075 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:57.368016    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:36:57.368075 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:57.368029    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:36:57.374949 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:57.374931    2568 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:36:57.391039 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:57.391014    2568 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:36:57.419765 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:57.419739    2568 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:36:57.449070 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:57.448994    2568 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:36:57.451458 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:57.451442    2568 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:36:57.506120 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:57.506094    2568 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:36:57.661964 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:57.661939    2568 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:36:57.826609 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:57.826576    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d836aa5280c51c365d2d598465f9d5c.slice/crio-3e103fa82ae87141bdaa8d02c8f8a91a33635d2868773802478d7187e4ab2200 WatchSource:0}: Error finding container 3e103fa82ae87141bdaa8d02c8f8a91a33635d2868773802478d7187e4ab2200: Status 404 returned error can't find the container with id 3e103fa82ae87141bdaa8d02c8f8a91a33635d2868773802478d7187e4ab2200
Oct 30 21:36:57.827097 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:57.827070    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85a4f26b44393e519ecc5f1c6d0f8225.slice/crio-f8de11d8a1076b1f125776dfbe8032416f0c5971e1bb947947ba43aa31611d31 WatchSource:0}: Error finding container f8de11d8a1076b1f125776dfbe8032416f0c5971e1bb947947ba43aa31611d31: Status 404 returned error can't find the container with id f8de11d8a1076b1f125776dfbe8032416f0c5971e1bb947947ba43aa31611d31
Oct 30 21:36:57.833986 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:57.833962    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:36:58.062285 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.062247    2568 apiserver.go:52] "Watching apiserver"
Oct 30 21:36:58.090616 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.090536    2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 30 21:36:58.159224 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.159192    2568 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 21:36:58.159522 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.159502    2568 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-ggqpf","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal","openshift-network-operator/iptables-alerter-2wb5p","openshift-ovn-kubernetes/ovnkube-node-wzntd","kube-system/global-pull-secret-syncer-z9h2r","kube-system/kube-apiserver-proxy-ip-10-0-139-46.us-west-2.compute.internal","openshift-cluster-node-tuning-operator/tuned-f9gtx","openshift-dns/node-resolver-dbpfd","openshift-image-registry/node-ca-grk5c","kube-system/konnectivity-agent-4vb42","openshift-multus/network-metrics-daemon-l9kxl","openshift-network-diagnostics/network-check-target-wpc6f","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9","openshift-multus/multus-additional-cni-plugins-4frz9"]
Oct 30 21:36:58.163456 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.163441    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.165574 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.165549    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:36:58.168057 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.167861    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2wb5p"
Oct 30 21:36:58.170812 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.170790    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.173007 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.172988    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:36:58.173101 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:58.173079    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9h2r" podUID="6e615be3-41d7-4f82-90d7-7ce71ed9190b"
Oct 30 21:36:58.175756 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.175687    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4frz9"
Oct 30 21:36:58.178190 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.178168    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-grk5c"
Oct 30 21:36:58.181282 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.181265    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.183664 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.183647    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dbpfd"
Oct 30 21:36:58.185888 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.185874    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4vb42"
Oct 30 21:36:58.188206 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.188189    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9kxl"
Oct 30 21:36:58.188333 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:58.188286    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9kxl" podUID="7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2"
Oct 30 21:36:58.189399 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.189385    2568 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:36:58.190586 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.190570    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wpc6f"
Oct 30 21:36:58.190653 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:58.190633    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wpc6f" podUID="9eefa84c-5304-4c2a-89cf-3304097ecf42"
Oct 30 21:36:58.201631 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.201607    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-etc-sysctl-conf\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.201708 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.201644    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/457fd859-0880-43f8-b712-2fd90c9debb0-etc-tuned\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.201708 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.201674    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cb8bde01-2bc6-4301-aa2f-2858158bd54e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4frz9\" (UID: \"cb8bde01-2bc6-4301-aa2f-2858158bd54e\") " pod="openshift-multus/multus-additional-cni-plugins-4frz9"
Oct 30 21:36:58.201789 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.201775    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-device-dir\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:36:58.201836 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.201821    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-host-var-lib-cni-multus\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.201877 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.201848    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-574vn\" (UniqueName: \"kubernetes.io/projected/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-kube-api-access-574vn\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:36:58.201877 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.201864    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-kubelet\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.201946 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.201896    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-multus-conf-dir\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.201946 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.201932    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxmv7\" (UniqueName: \"kubernetes.io/projected/9eefa84c-5304-4c2a-89cf-3304097ecf42-kube-api-access-gxmv7\") pod \"network-check-target-wpc6f\" (UID: \"9eefa84c-5304-4c2a-89cf-3304097ecf42\") " pod="openshift-network-diagnostics/network-check-target-wpc6f"
Oct 30 21:36:58.202005 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.201971    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-slash\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.202033 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.202008    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-node-log\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.202061 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.202035    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/07b45ded-5ba8-49dc-b6ab-c170ca47af9b-host-slash\") pod \"iptables-alerter-2wb5p\" (UID: \"07b45ded-5ba8-49dc-b6ab-c170ca47af9b\") " pod="openshift-network-operator/iptables-alerter-2wb5p"
Oct 30 21:36:58.202061 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.202055    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cb8bde01-2bc6-4301-aa2f-2858158bd54e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4frz9\" (UID: \"cb8bde01-2bc6-4301-aa2f-2858158bd54e\") " pod="openshift-multus/multus-additional-cni-plugins-4frz9"
Oct 30 21:36:58.202119 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.202088    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-sys\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.202161 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.202119    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-run-netns\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.202212 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.202193    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-run-openvswitch\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.202259 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.202230    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-run-ovn-kubernetes\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.202298 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.202285    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-cni-netd\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.202330 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.202319    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7lhk\" (UniqueName: \"kubernetes.io/projected/2360fddc-52a5-42a5-82ca-a05aff84d217-kube-api-access-v7lhk\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.202360 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.202337    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-host-var-lib-cni-bin\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.202391 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.202375    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-etc-kubernetes\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.202424 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.202401    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-etc-systemd\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.202457 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.202431    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-systemd-units\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.202517 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.202495    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-log-socket\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.202564 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.202538    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/948affe7-aaf8-4d3e-a515-ed9552d19d90-host\") pod \"node-ca-grk5c\" (UID: \"948affe7-aaf8-4d3e-a515-ed9552d19d90\") " pod="openshift-image-registry/node-ca-grk5c"
Oct 30 21:36:58.202603 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.202562    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/726700d7-c252-479e-abdd-18c0573102d1-multus-daemon-config\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.202603 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.202584    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/457fd859-0880-43f8-b712-2fd90c9debb0-tmp\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.202672 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.202630    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-cnibin\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.202672 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.202658    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-host-run-netns\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.202743 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.202686    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6e615be3-41d7-4f82-90d7-7ce71ed9190b-kubelet-config\") pod \"global-pull-secret-syncer-z9h2r\" (UID: \"6e615be3-41d7-4f82-90d7-7ce71ed9190b\") " pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:36:58.202743 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.202705    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6e615be3-41d7-4f82-90d7-7ce71ed9190b-dbus\") pod \"global-pull-secret-syncer-z9h2r\" (UID: \"6e615be3-41d7-4f82-90d7-7ce71ed9190b\") " pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:36:58.202743 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.202725    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:36:58.202743 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.202739    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-sys-fs\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:36:58.202901 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.202784    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-etc-openvswitch\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.202901 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.202823    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-etc-modprobe-d\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.202901 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.202859    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-etc-sysconfig\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.203015 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.202933    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/07b45ded-5ba8-49dc-b6ab-c170ca47af9b-iptables-alerter-script\") pod \"iptables-alerter-2wb5p\" (UID: \"07b45ded-5ba8-49dc-b6ab-c170ca47af9b\") " pod="openshift-network-operator/iptables-alerter-2wb5p"
Oct 30 21:36:58.203015 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.202965    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfcrl\" (UniqueName: \"kubernetes.io/projected/07b45ded-5ba8-49dc-b6ab-c170ca47af9b-kube-api-access-nfcrl\") pod \"iptables-alerter-2wb5p\" (UID: \"07b45ded-5ba8-49dc-b6ab-c170ca47af9b\") " pod="openshift-network-operator/iptables-alerter-2wb5p"
Oct 30 21:36:58.203015 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.202991    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cb8bde01-2bc6-4301-aa2f-2858158bd54e-cni-binary-copy\") pod \"multus-additional-cni-plugins-4frz9\" (UID: \"cb8bde01-2bc6-4301-aa2f-2858158bd54e\") " pod="openshift-multus/multus-additional-cni-plugins-4frz9"
Oct 30 21:36:58.203123 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.203042    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmlvz\" (UniqueName: \"kubernetes.io/projected/cb8bde01-2bc6-4301-aa2f-2858158bd54e-kube-api-access-qmlvz\") pod \"multus-additional-cni-plugins-4frz9\" (UID: \"cb8bde01-2bc6-4301-aa2f-2858158bd54e\") " pod="openshift-multus/multus-additional-cni-plugins-4frz9"
Oct 30 21:36:58.203123 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.203068    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrvv4\" (UniqueName: \"kubernetes.io/projected/909906d5-5049-4742-9b5a-b9c03ed5cc39-kube-api-access-vrvv4\") pod \"node-resolver-dbpfd\" (UID: \"909906d5-5049-4742-9b5a-b9c03ed5cc39\") " pod="openshift-dns/node-resolver-dbpfd"
Oct 30 21:36:58.203123 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.203106    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-var-lib-openvswitch\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.203209 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.203127    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-metrics-certs\") pod \"network-metrics-daemon-l9kxl\" (UID: \"7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2\") " pod="openshift-multus/network-metrics-daemon-l9kxl"
Oct 30 21:36:58.203209 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.203143    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-socket-dir\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:36:58.203209 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.203157    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wv5hx\" (UniqueName: \"kubernetes.io/projected/948affe7-aaf8-4d3e-a515-ed9552d19d90-kube-api-access-wv5hx\") pod \"node-ca-grk5c\" (UID: \"948affe7-aaf8-4d3e-a515-ed9552d19d90\") " pod="openshift-image-registry/node-ca-grk5c"
Oct 30 21:36:58.203209 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.203173    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-system-cni-dir\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.203343 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.203205    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-run-systemd\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.203343 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.203265    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6d9d43c2-40da-47a8-adb4-d3ba378d77d9-konnectivity-ca\") pod \"konnectivity-agent-4vb42\" (UID: \"6d9d43c2-40da-47a8-adb4-d3ba378d77d9\") " pod="kube-system/konnectivity-agent-4vb42"
Oct 30 21:36:58.203343 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.203299    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lnqk\" (UniqueName: \"kubernetes.io/projected/457fd859-0880-43f8-b712-2fd90c9debb0-kube-api-access-6lnqk\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.203343 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.203319    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e615be3-41d7-4f82-90d7-7ce71ed9190b-original-pull-secret\") pod \"global-pull-secret-syncer-z9h2r\" (UID: \"6e615be3-41d7-4f82-90d7-7ce71ed9190b\") " pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:36:58.203480 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.203347    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/948affe7-aaf8-4d3e-a515-ed9552d19d90-serviceca\") pod \"node-ca-grk5c\" (UID: \"948affe7-aaf8-4d3e-a515-ed9552d19d90\") " pod="openshift-image-registry/node-ca-grk5c"
Oct 30 21:36:58.203480 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.203383    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-run\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.203480 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.203411    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxsh7\" (UniqueName: \"kubernetes.io/projected/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-kube-api-access-jxsh7\") pod \"network-metrics-daemon-l9kxl\" (UID: \"7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2\") " pod="openshift-multus/network-metrics-daemon-l9kxl"
Oct 30 21:36:58.203480 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.203437    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-registration-dir\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:36:58.203605 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.203482    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-cni-bin\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.203605 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.203504    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2360fddc-52a5-42a5-82ca-a05aff84d217-ovn-node-metrics-cert\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.203605 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.203525    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cb8bde01-2bc6-4301-aa2f-2858158bd54e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4frz9\" (UID: \"cb8bde01-2bc6-4301-aa2f-2858158bd54e\") " pod="openshift-multus/multus-additional-cni-plugins-4frz9"
Oct 30 21:36:58.203605 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.203546    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/909906d5-5049-4742-9b5a-b9c03ed5cc39-hosts-file\") pod \"node-resolver-dbpfd\" (UID: \"909906d5-5049-4742-9b5a-b9c03ed5cc39\") " pod="openshift-dns/node-resolver-dbpfd"
Oct 30 21:36:58.203605 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.203567    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-lib-modules\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.203605 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.203584    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-multus-cni-dir\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.203763 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.203604    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.203763 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.203631    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/726700d7-c252-479e-abdd-18c0573102d1-cni-binary-copy\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.203763 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.203647    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb8bde01-2bc6-4301-aa2f-2858158bd54e-system-cni-dir\") pod \"multus-additional-cni-plugins-4frz9\" (UID: \"cb8bde01-2bc6-4301-aa2f-2858158bd54e\") " pod="openshift-multus/multus-additional-cni-plugins-4frz9"
Oct 30 21:36:58.203763 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.203668    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4xm6\" (UniqueName: \"kubernetes.io/projected/726700d7-c252-479e-abdd-18c0573102d1-kube-api-access-z4xm6\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.203763 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.203690    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6d9d43c2-40da-47a8-adb4-d3ba378d77d9-agent-certs\") pod \"konnectivity-agent-4vb42\" (UID: \"6d9d43c2-40da-47a8-adb4-d3ba378d77d9\") " pod="kube-system/konnectivity-agent-4vb42"
Oct 30 21:36:58.203763 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.203711    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-etc-kubernetes\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.203763 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.203735    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/909906d5-5049-4742-9b5a-b9c03ed5cc39-tmp-dir\") pod \"node-resolver-dbpfd\" (UID: \"909906d5-5049-4742-9b5a-b9c03ed5cc39\") " pod="openshift-dns/node-resolver-dbpfd"
Oct 30 21:36:58.203763 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.203751    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-run-ovn\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.203974 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.203785    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-etc-sysctl-d\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.203974 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.203807    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-var-lib-kubelet\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.203974 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.203827    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-host\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.203974 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.203843    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-host-run-k8s-cni-cncf-io\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.203974 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.203881    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-host-var-lib-kubelet\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.203974 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.203921    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-hostroot\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.204169 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.203969    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-host-run-multus-certs\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.204169 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.204038    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cb8bde01-2bc6-4301-aa2f-2858158bd54e-cnibin\") pod \"multus-additional-cni-plugins-4frz9\" (UID: \"cb8bde01-2bc6-4301-aa2f-2858158bd54e\") " pod="openshift-multus/multus-additional-cni-plugins-4frz9"
Oct 30 21:36:58.204169 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.204064    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-etc-selinux\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:36:58.204169 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.204089    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2360fddc-52a5-42a5-82ca-a05aff84d217-ovnkube-config\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.204169 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.204114    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2360fddc-52a5-42a5-82ca-a05aff84d217-env-overrides\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.204169 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.204155    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2360fddc-52a5-42a5-82ca-a05aff84d217-ovnkube-script-lib\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.204418 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.204177    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-multus-socket-dir-parent\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.212786 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.212756    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 21:36:58.212871 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.212831    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-p5bth\""
Oct 30 21:36:58.212871 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.212850    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:36:58.214424 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.214392    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 21:36:58.214506 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.214455    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 21:36:58.216781 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.214919    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gfsxs\""
Oct 30 21:36:58.220711 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.220677    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal" event={"ID":"85a4f26b44393e519ecc5f1c6d0f8225","Type":"ContainerStarted","Data":"f8de11d8a1076b1f125776dfbe8032416f0c5971e1bb947947ba43aa31611d31"}
Oct 30 21:36:58.221811 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.221789    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-46.us-west-2.compute.internal" event={"ID":"1d836aa5280c51c365d2d598465f9d5c","Type":"ContainerStarted","Data":"3e103fa82ae87141bdaa8d02c8f8a91a33635d2868773802478d7187e4ab2200"}
Oct 30 21:36:58.237954 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.237933    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 21:36:58.238035 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.238025    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 21:36:58.238404 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.238385    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hkmjr\""
Oct 30 21:36:58.239159 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.239135    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-b6pf6\""
Oct 30 21:36:58.239159 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.239142    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zlst6\""
Oct 30 21:36:58.239159 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.239154    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-j5bph\""
Oct 30 21:36:58.239385 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.239270    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-hcvt7\""
Oct 30 21:36:58.239385 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.239275    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-t84mq\""
Oct 30 21:36:58.239385 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.239277    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 21:36:58.239385 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.239279    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-lt5wb\""
Oct 30 21:36:58.239629 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.239615    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 21:36:58.240316 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.240304    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 21:36:58.255024 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.255007    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 21:36:58.259520 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.259502    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 21:36:58.259520 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.259515    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 21:36:58.259660 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.259530    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 21:36:58.259660 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.259562    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 21:36:58.261111 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.261093    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 21:36:58.261198 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.261126    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 21:36:58.261345 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.261332    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 21:36:58.261345 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.261334    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:36:58.285858 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.285821    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 21:36:58.285858 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.285844    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 21:36:58.286007 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.285939    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 21:36:58.286007 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.285958    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 21:36:58.286007 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.285986    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 21:36:58.286007 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.285989    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 21:36:58.286193 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.285996    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 21:36:58.286193 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.286037    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 21:36:58.286193 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.286066    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 21:36:58.286193 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.286090    2568 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 21:36:58.286193 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.286147    2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hnclw"
Oct 30 21:36:58.291560 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.291545    2568 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 21:36:58.304391 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.304359    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-etc-openvswitch\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.304532 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.304396    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-etc-modprobe-d\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.304532 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.304421    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-etc-sysconfig\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.304532 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.304444    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/07b45ded-5ba8-49dc-b6ab-c170ca47af9b-iptables-alerter-script\") pod \"iptables-alerter-2wb5p\" (UID: \"07b45ded-5ba8-49dc-b6ab-c170ca47af9b\") " pod="openshift-network-operator/iptables-alerter-2wb5p"
Oct 30 21:36:58.304532 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.304467    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfcrl\" (UniqueName: \"kubernetes.io/projected/07b45ded-5ba8-49dc-b6ab-c170ca47af9b-kube-api-access-nfcrl\") pod \"iptables-alerter-2wb5p\" (UID: \"07b45ded-5ba8-49dc-b6ab-c170ca47af9b\") " pod="openshift-network-operator/iptables-alerter-2wb5p"
Oct 30 21:36:58.304532 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.304471    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-etc-openvswitch\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.304532 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.304489    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cb8bde01-2bc6-4301-aa2f-2858158bd54e-cni-binary-copy\") pod \"multus-additional-cni-plugins-4frz9\" (UID: \"cb8bde01-2bc6-4301-aa2f-2858158bd54e\") " pod="openshift-multus/multus-additional-cni-plugins-4frz9"
Oct 30 21:36:58.304532 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.304497    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-etc-sysconfig\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.304532 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.304528    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmlvz\" (UniqueName: \"kubernetes.io/projected/cb8bde01-2bc6-4301-aa2f-2858158bd54e-kube-api-access-qmlvz\") pod \"multus-additional-cni-plugins-4frz9\" (UID: \"cb8bde01-2bc6-4301-aa2f-2858158bd54e\") " pod="openshift-multus/multus-additional-cni-plugins-4frz9"
Oct 30 21:36:58.304890 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.304545    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-etc-modprobe-d\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.304890 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.304559    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrvv4\" (UniqueName: \"kubernetes.io/projected/909906d5-5049-4742-9b5a-b9c03ed5cc39-kube-api-access-vrvv4\") pod \"node-resolver-dbpfd\" (UID: \"909906d5-5049-4742-9b5a-b9c03ed5cc39\") " pod="openshift-dns/node-resolver-dbpfd"
Oct 30 21:36:58.304890 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.304585    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-var-lib-openvswitch\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.304890 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.304609    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-metrics-certs\") pod \"network-metrics-daemon-l9kxl\" (UID: \"7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2\") " pod="openshift-multus/network-metrics-daemon-l9kxl"
Oct 30 21:36:58.304890 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.304631    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-socket-dir\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:36:58.304890 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.304654    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wv5hx\" (UniqueName: \"kubernetes.io/projected/948affe7-aaf8-4d3e-a515-ed9552d19d90-kube-api-access-wv5hx\") pod \"node-ca-grk5c\" (UID: \"948affe7-aaf8-4d3e-a515-ed9552d19d90\") " pod="openshift-image-registry/node-ca-grk5c"
Oct 30 21:36:58.304890 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.304662    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-var-lib-openvswitch\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.304890 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.304678    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-system-cni-dir\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.304890 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.304709    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-run-systemd\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.304890 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.304729    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-system-cni-dir\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.304890 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.304739    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6d9d43c2-40da-47a8-adb4-d3ba378d77d9-konnectivity-ca\") pod \"konnectivity-agent-4vb42\" (UID: \"6d9d43c2-40da-47a8-adb4-d3ba378d77d9\") " pod="kube-system/konnectivity-agent-4vb42"
Oct 30 21:36:58.304890 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.304766    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lnqk\" (UniqueName: \"kubernetes.io/projected/457fd859-0880-43f8-b712-2fd90c9debb0-kube-api-access-6lnqk\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.304890 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.304790    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e615be3-41d7-4f82-90d7-7ce71ed9190b-original-pull-secret\") pod \"global-pull-secret-syncer-z9h2r\" (UID: \"6e615be3-41d7-4f82-90d7-7ce71ed9190b\") " pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:36:58.304890 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.304814    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/948affe7-aaf8-4d3e-a515-ed9552d19d90-serviceca\") pod \"node-ca-grk5c\" (UID: \"948affe7-aaf8-4d3e-a515-ed9552d19d90\") " pod="openshift-image-registry/node-ca-grk5c"
Oct 30 21:36:58.304890 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:58.304841    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:36:58.305613 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.304855    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-socket-dir\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:36:58.305613 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.304910    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-run-systemd\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.305613 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:58.304947    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-metrics-certs podName:7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2 nodeName:}" failed. No retries permitted until 2026-10-30 21:36:58.804924151 +0000 UTC m=+3.131416703 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-metrics-certs") pod "network-metrics-daemon-l9kxl" (UID: "7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:36:58.305613 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305015    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-run\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.305613 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305056    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxsh7\" (UniqueName: \"kubernetes.io/projected/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-kube-api-access-jxsh7\") pod \"network-metrics-daemon-l9kxl\" (UID: \"7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2\") " pod="openshift-multus/network-metrics-daemon-l9kxl"
Oct 30 21:36:58.305613 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305084    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-registration-dir\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:36:58.305613 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:58.305106    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:36:58.305613 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305148    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-cni-bin\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.305613 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:58.305166    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e615be3-41d7-4f82-90d7-7ce71ed9190b-original-pull-secret podName:6e615be3-41d7-4f82-90d7-7ce71ed9190b nodeName:}" failed. No retries permitted until 2026-10-30 21:36:58.805147246 +0000 UTC m=+3.131639792 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e615be3-41d7-4f82-90d7-7ce71ed9190b-original-pull-secret") pod "global-pull-secret-syncer-z9h2r" (UID: "6e615be3-41d7-4f82-90d7-7ce71ed9190b") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:36:58.305613 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305108    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-cni-bin\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.305613 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305199    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-registration-dir\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:36:58.305613 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305205    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2360fddc-52a5-42a5-82ca-a05aff84d217-ovn-node-metrics-cert\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.305613 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305215    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/07b45ded-5ba8-49dc-b6ab-c170ca47af9b-iptables-alerter-script\") pod \"iptables-alerter-2wb5p\" (UID: \"07b45ded-5ba8-49dc-b6ab-c170ca47af9b\") " pod="openshift-network-operator/iptables-alerter-2wb5p"
Oct 30 21:36:58.305613 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305233    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cb8bde01-2bc6-4301-aa2f-2858158bd54e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4frz9\" (UID: \"cb8bde01-2bc6-4301-aa2f-2858158bd54e\") " pod="openshift-multus/multus-additional-cni-plugins-4frz9"
Oct 30 21:36:58.305613 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305286    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/909906d5-5049-4742-9b5a-b9c03ed5cc39-hosts-file\") pod \"node-resolver-dbpfd\" (UID: \"909906d5-5049-4742-9b5a-b9c03ed5cc39\") " pod="openshift-dns/node-resolver-dbpfd"
Oct 30 21:36:58.305613 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305296    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cb8bde01-2bc6-4301-aa2f-2858158bd54e-cni-binary-copy\") pod \"multus-additional-cni-plugins-4frz9\" (UID: \"cb8bde01-2bc6-4301-aa2f-2858158bd54e\") " pod="openshift-multus/multus-additional-cni-plugins-4frz9"
Oct 30 21:36:58.306380 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305333    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/948affe7-aaf8-4d3e-a515-ed9552d19d90-serviceca\") pod \"node-ca-grk5c\" (UID: \"948affe7-aaf8-4d3e-a515-ed9552d19d90\") " pod="openshift-image-registry/node-ca-grk5c"
Oct 30 21:36:58.306380 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305343    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/909906d5-5049-4742-9b5a-b9c03ed5cc39-hosts-file\") pod \"node-resolver-dbpfd\" (UID: \"909906d5-5049-4742-9b5a-b9c03ed5cc39\") " pod="openshift-dns/node-resolver-dbpfd"
Oct 30 21:36:58.306380 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305346    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-run\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.306380 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305372    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-lib-modules\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.306380 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305395    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-multus-cni-dir\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.306380 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305427    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.306380 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305453    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/726700d7-c252-479e-abdd-18c0573102d1-cni-binary-copy\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.306380 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305473    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb8bde01-2bc6-4301-aa2f-2858158bd54e-system-cni-dir\") pod \"multus-additional-cni-plugins-4frz9\" (UID: \"cb8bde01-2bc6-4301-aa2f-2858158bd54e\") " pod="openshift-multus/multus-additional-cni-plugins-4frz9"
Oct 30 21:36:58.306380 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305498    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4xm6\" (UniqueName: \"kubernetes.io/projected/726700d7-c252-479e-abdd-18c0573102d1-kube-api-access-z4xm6\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.306380 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305512    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-lib-modules\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.306380 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305526    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6d9d43c2-40da-47a8-adb4-d3ba378d77d9-agent-certs\") pod \"konnectivity-agent-4vb42\" (UID: \"6d9d43c2-40da-47a8-adb4-d3ba378d77d9\") " pod="kube-system/konnectivity-agent-4vb42"
Oct 30 21:36:58.306380 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305558    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-etc-kubernetes\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.306380 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305566    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6d9d43c2-40da-47a8-adb4-d3ba378d77d9-konnectivity-ca\") pod \"konnectivity-agent-4vb42\" (UID: \"6d9d43c2-40da-47a8-adb4-d3ba378d77d9\") " pod="kube-system/konnectivity-agent-4vb42"
Oct 30 21:36:58.306380 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305577    2568 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 21:36:58.306380 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305570    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-multus-cni-dir\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.306380 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305607    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.306380 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305630    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-etc-kubernetes\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.306380 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305633    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb8bde01-2bc6-4301-aa2f-2858158bd54e-system-cni-dir\") pod \"multus-additional-cni-plugins-4frz9\" (UID: \"cb8bde01-2bc6-4301-aa2f-2858158bd54e\") " pod="openshift-multus/multus-additional-cni-plugins-4frz9"
Oct 30 21:36:58.307192 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305584    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/909906d5-5049-4742-9b5a-b9c03ed5cc39-tmp-dir\") pod \"node-resolver-dbpfd\" (UID: \"909906d5-5049-4742-9b5a-b9c03ed5cc39\") " pod="openshift-dns/node-resolver-dbpfd"
Oct 30 21:36:58.307192 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305681    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-run-ovn\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.307192 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305708    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-etc-sysctl-d\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.307192 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305731    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-var-lib-kubelet\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.307192 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305753    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-host\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.307192 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305776    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-host-run-k8s-cni-cncf-io\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.307192 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305804    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-host-var-lib-kubelet\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.307192 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305821    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-run-ovn\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.307192 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305829    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-hostroot\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.307192 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305853    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-host-run-multus-certs\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.307192 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305858    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cb8bde01-2bc6-4301-aa2f-2858158bd54e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4frz9\" (UID: \"cb8bde01-2bc6-4301-aa2f-2858158bd54e\") " pod="openshift-multus/multus-additional-cni-plugins-4frz9"
Oct 30 21:36:58.307192 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305876    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cb8bde01-2bc6-4301-aa2f-2858158bd54e-cnibin\") pod \"multus-additional-cni-plugins-4frz9\" (UID: \"cb8bde01-2bc6-4301-aa2f-2858158bd54e\") " pod="openshift-multus/multus-additional-cni-plugins-4frz9"
Oct 30 21:36:58.307192 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305866    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-host-run-k8s-cni-cncf-io\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.307192 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305900    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-etc-selinux\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:36:58.307192 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305904    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-var-lib-kubelet\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.307192 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305924    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2360fddc-52a5-42a5-82ca-a05aff84d217-ovnkube-config\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.307192 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305932    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-hostroot\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.307192 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305944    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/909906d5-5049-4742-9b5a-b9c03ed5cc39-tmp-dir\") pod \"node-resolver-dbpfd\" (UID: \"909906d5-5049-4742-9b5a-b9c03ed5cc39\") " pod="openshift-dns/node-resolver-dbpfd"
Oct 30 21:36:58.307838 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305950    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-host\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.307838 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305950    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2360fddc-52a5-42a5-82ca-a05aff84d217-env-overrides\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.307838 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.305983    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cb8bde01-2bc6-4301-aa2f-2858158bd54e-cnibin\") pod \"multus-additional-cni-plugins-4frz9\" (UID: \"cb8bde01-2bc6-4301-aa2f-2858158bd54e\") " pod="openshift-multus/multus-additional-cni-plugins-4frz9"
Oct 30 21:36:58.307838 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306002    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2360fddc-52a5-42a5-82ca-a05aff84d217-ovnkube-script-lib\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.307838 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306025    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-multus-socket-dir-parent\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.307838 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306031    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-etc-sysctl-d\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.307838 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306045    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-etc-sysctl-conf\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.307838 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306047    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-etc-selinux\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:36:58.307838 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306064    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/457fd859-0880-43f8-b712-2fd90c9debb0-etc-tuned\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.307838 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306076    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-host-var-lib-kubelet\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.307838 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306090    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cb8bde01-2bc6-4301-aa2f-2858158bd54e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4frz9\" (UID: \"cb8bde01-2bc6-4301-aa2f-2858158bd54e\") " pod="openshift-multus/multus-additional-cni-plugins-4frz9"
Oct 30 21:36:58.307838 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306113    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-device-dir\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:36:58.307838 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306120    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-host-run-multus-certs\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.307838 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306165    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-device-dir\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:36:58.307838 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306181    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/726700d7-c252-479e-abdd-18c0573102d1-cni-binary-copy\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.307838 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306230    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-etc-sysctl-conf\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.307838 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306273    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-host-var-lib-cni-multus\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.308440 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306305    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-574vn\" (UniqueName: \"kubernetes.io/projected/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-kube-api-access-574vn\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:36:58.308440 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306324    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-multus-socket-dir-parent\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.308440 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306333    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-kubelet\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.308440 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306344    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-host-var-lib-cni-multus\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.308440 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306366    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-multus-conf-dir\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.308440 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306369    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-kubelet\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.308440 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306392    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxmv7\" (UniqueName: \"kubernetes.io/projected/9eefa84c-5304-4c2a-89cf-3304097ecf42-kube-api-access-gxmv7\") pod \"network-check-target-wpc6f\" (UID: \"9eefa84c-5304-4c2a-89cf-3304097ecf42\") " pod="openshift-network-diagnostics/network-check-target-wpc6f"
Oct 30 21:36:58.308440 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306412    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-multus-conf-dir\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.308440 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306418    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-slash\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.308440 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306442    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-node-log\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.308440 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306488    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/07b45ded-5ba8-49dc-b6ab-c170ca47af9b-host-slash\") pod \"iptables-alerter-2wb5p\" (UID: \"07b45ded-5ba8-49dc-b6ab-c170ca47af9b\") " pod="openshift-network-operator/iptables-alerter-2wb5p"
Oct 30 21:36:58.308440 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306513    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cb8bde01-2bc6-4301-aa2f-2858158bd54e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4frz9\" (UID: \"cb8bde01-2bc6-4301-aa2f-2858158bd54e\") " pod="openshift-multus/multus-additional-cni-plugins-4frz9"
Oct 30 21:36:58.308440 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306537    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-sys\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.308440 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306545    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-node-log\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.308440 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306564    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-run-netns\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.308440 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306586    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-slash\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.308440 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306588    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-run-openvswitch\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.308916 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306624    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-run-ovn-kubernetes\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.308916 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306672    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-sys\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.308916 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306681    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-cni-netd\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.308916 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306696    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/07b45ded-5ba8-49dc-b6ab-c170ca47af9b-host-slash\") pod \"iptables-alerter-2wb5p\" (UID: \"07b45ded-5ba8-49dc-b6ab-c170ca47af9b\") " pod="openshift-network-operator/iptables-alerter-2wb5p"
Oct 30 21:36:58.308916 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306709    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7lhk\" (UniqueName: \"kubernetes.io/projected/2360fddc-52a5-42a5-82ca-a05aff84d217-kube-api-access-v7lhk\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.308916 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306716    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-run-netns\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.308916 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306736    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-host-var-lib-cni-bin\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.308916 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306761    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-etc-kubernetes\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.308916 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306768    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-run-ovn-kubernetes\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.308916 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306784    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-etc-systemd\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.308916 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306813    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-host-var-lib-cni-bin\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.308916 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306827    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-systemd-units\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.308916 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306850    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-cni-netd\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.308916 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306854    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-log-socket\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.308916 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306854    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-etc-kubernetes\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.308916 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306626    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-run-openvswitch\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.308916 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306883    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/948affe7-aaf8-4d3e-a515-ed9552d19d90-host\") pod \"node-ca-grk5c\" (UID: \"948affe7-aaf8-4d3e-a515-ed9552d19d90\") " pod="openshift-image-registry/node-ca-grk5c"
Oct 30 21:36:58.308916 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306897    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-etc-systemd\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.309470 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306904    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-systemd-units\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.309470 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306910    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/726700d7-c252-479e-abdd-18c0573102d1-multus-daemon-config\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.309470 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306915    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-log-socket\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.309470 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306948    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/948affe7-aaf8-4d3e-a515-ed9552d19d90-host\") pod \"node-ca-grk5c\" (UID: \"948affe7-aaf8-4d3e-a515-ed9552d19d90\") " pod="openshift-image-registry/node-ca-grk5c"
Oct 30 21:36:58.309470 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306972    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/457fd859-0880-43f8-b712-2fd90c9debb0-tmp\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.309470 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.306994    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-cnibin\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.309470 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.307015    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-host-run-netns\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.309470 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.307041    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6e615be3-41d7-4f82-90d7-7ce71ed9190b-kubelet-config\") pod \"global-pull-secret-syncer-z9h2r\" (UID: \"6e615be3-41d7-4f82-90d7-7ce71ed9190b\") " pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:36:58.309470 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.307064    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6e615be3-41d7-4f82-90d7-7ce71ed9190b-dbus\") pod \"global-pull-secret-syncer-z9h2r\" (UID: \"6e615be3-41d7-4f82-90d7-7ce71ed9190b\") " pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:36:58.309470 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.307087    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:36:58.309470 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.307101    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2360fddc-52a5-42a5-82ca-a05aff84d217-env-overrides\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.309470 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.307109    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-sys-fs\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:36:58.309470 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.307151    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-sys-fs\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:36:58.309470 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.307169    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cb8bde01-2bc6-4301-aa2f-2858158bd54e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4frz9\" (UID: \"cb8bde01-2bc6-4301-aa2f-2858158bd54e\") " pod="openshift-multus/multus-additional-cni-plugins-4frz9"
Oct 30 21:36:58.309470 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.307223    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-cnibin\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.309470 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.307261    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-host-run-netns\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.309470 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.307287    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6e615be3-41d7-4f82-90d7-7ce71ed9190b-kubelet-config\") pod \"global-pull-secret-syncer-z9h2r\" (UID: \"6e615be3-41d7-4f82-90d7-7ce71ed9190b\") " pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:36:58.309470 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.307338    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:36:58.310093 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.307355    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6e615be3-41d7-4f82-90d7-7ce71ed9190b-dbus\") pod \"global-pull-secret-syncer-z9h2r\" (UID: \"6e615be3-41d7-4f82-90d7-7ce71ed9190b\") " pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:36:58.310093 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.307397    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/726700d7-c252-479e-abdd-18c0573102d1-multus-daemon-config\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.310093 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.307989    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2360fddc-52a5-42a5-82ca-a05aff84d217-ovnkube-script-lib\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.310093 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.308192    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cb8bde01-2bc6-4301-aa2f-2858158bd54e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4frz9\" (UID: \"cb8bde01-2bc6-4301-aa2f-2858158bd54e\") " pod="openshift-multus/multus-additional-cni-plugins-4frz9"
Oct 30 21:36:58.310093 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.308381    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2360fddc-52a5-42a5-82ca-a05aff84d217-ovnkube-config\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.310093 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.309288    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/457fd859-0880-43f8-b712-2fd90c9debb0-etc-tuned\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.310093 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.309305    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/457fd859-0880-43f8-b712-2fd90c9debb0-tmp\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.310093 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.309526    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2360fddc-52a5-42a5-82ca-a05aff84d217-ovn-node-metrics-cert\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.310093 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.309646    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6d9d43c2-40da-47a8-adb4-d3ba378d77d9-agent-certs\") pod \"konnectivity-agent-4vb42\" (UID: \"6d9d43c2-40da-47a8-adb4-d3ba378d77d9\") " pod="kube-system/konnectivity-agent-4vb42"
Oct 30 21:36:58.362092 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:58.362007    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:36:58.362092 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:58.362052    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:36:58.362092 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:58.362068    2568 projected.go:196] Error preparing data for projected volume kube-api-access-gxmv7 for pod openshift-network-diagnostics/network-check-target-wpc6f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:36:58.362365 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:58.362207    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9eefa84c-5304-4c2a-89cf-3304097ecf42-kube-api-access-gxmv7 podName:9eefa84c-5304-4c2a-89cf-3304097ecf42 nodeName:}" failed. No retries permitted until 2026-10-30 21:36:58.862183696 +0000 UTC m=+3.188676253 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-gxmv7" (UniqueName: "kubernetes.io/projected/9eefa84c-5304-4c2a-89cf-3304097ecf42-kube-api-access-gxmv7") pod "network-check-target-wpc6f" (UID: "9eefa84c-5304-4c2a-89cf-3304097ecf42") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:36:58.364113 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.364088    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lnqk\" (UniqueName: \"kubernetes.io/projected/457fd859-0880-43f8-b712-2fd90c9debb0-kube-api-access-6lnqk\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.364205 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.364096    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4xm6\" (UniqueName: \"kubernetes.io/projected/726700d7-c252-479e-abdd-18c0573102d1-kube-api-access-z4xm6\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.364205 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.364153    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wv5hx\" (UniqueName: \"kubernetes.io/projected/948affe7-aaf8-4d3e-a515-ed9552d19d90-kube-api-access-wv5hx\") pod \"node-ca-grk5c\" (UID: \"948affe7-aaf8-4d3e-a515-ed9552d19d90\") " pod="openshift-image-registry/node-ca-grk5c"
Oct 30 21:36:58.364205 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.364143    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrvv4\" (UniqueName: \"kubernetes.io/projected/909906d5-5049-4742-9b5a-b9c03ed5cc39-kube-api-access-vrvv4\") pod \"node-resolver-dbpfd\" (UID: \"909906d5-5049-4742-9b5a-b9c03ed5cc39\") " pod="openshift-dns/node-resolver-dbpfd"
Oct 30 21:36:58.364205 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.364185    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmlvz\" (UniqueName: \"kubernetes.io/projected/cb8bde01-2bc6-4301-aa2f-2858158bd54e-kube-api-access-qmlvz\") pod \"multus-additional-cni-plugins-4frz9\" (UID: \"cb8bde01-2bc6-4301-aa2f-2858158bd54e\") " pod="openshift-multus/multus-additional-cni-plugins-4frz9"
Oct 30 21:36:58.364381 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.364362    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfcrl\" (UniqueName: \"kubernetes.io/projected/07b45ded-5ba8-49dc-b6ab-c170ca47af9b-kube-api-access-nfcrl\") pod \"iptables-alerter-2wb5p\" (UID: \"07b45ded-5ba8-49dc-b6ab-c170ca47af9b\") " pod="openshift-network-operator/iptables-alerter-2wb5p"
Oct 30 21:36:58.364906 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.364885    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxsh7\" (UniqueName: \"kubernetes.io/projected/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-kube-api-access-jxsh7\") pod \"network-metrics-daemon-l9kxl\" (UID: \"7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2\") " pod="openshift-multus/network-metrics-daemon-l9kxl"
Oct 30 21:36:58.365319 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.365304    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7lhk\" (UniqueName: \"kubernetes.io/projected/2360fddc-52a5-42a5-82ca-a05aff84d217-kube-api-access-v7lhk\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.365971 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.365957    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-574vn\" (UniqueName: \"kubernetes.io/projected/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-kube-api-access-574vn\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:36:58.475573 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.475526    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:36:58.485003 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.484976    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:36:58.485482 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:58.485456    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod457fd859_0880_43f8_b712_2fd90c9debb0.slice/crio-8cbeffb016005bf058c1408477fa97d052e5e650265e29ebc098011a6888d1e8 WatchSource:0}: Error finding container 8cbeffb016005bf058c1408477fa97d052e5e650265e29ebc098011a6888d1e8: Status 404 returned error can't find the container with id 8cbeffb016005bf058c1408477fa97d052e5e650265e29ebc098011a6888d1e8
Oct 30 21:36:58.488861 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.488844    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2wb5p"
Oct 30 21:36:58.493569 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:58.493545    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b99dea5_32ac_4c1c_af68_b5d2b7af23c2.slice/crio-c4c22dc33e25512d0c3ae645987661746b0c1ac84cf568e3250994918ee72d31 WatchSource:0}: Error finding container c4c22dc33e25512d0c3ae645987661746b0c1ac84cf568e3250994918ee72d31: Status 404 returned error can't find the container with id c4c22dc33e25512d0c3ae645987661746b0c1ac84cf568e3250994918ee72d31
Oct 30 21:36:58.497326 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:58.497306    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07b45ded_5ba8_49dc_b6ab_c170ca47af9b.slice/crio-9d70bbf91a12f4450deb83490604353f3607520ccdbe175a3ecc4b6a93654eef WatchSource:0}: Error finding container 9d70bbf91a12f4450deb83490604353f3607520ccdbe175a3ecc4b6a93654eef: Status 404 returned error can't find the container with id 9d70bbf91a12f4450deb83490604353f3607520ccdbe175a3ecc4b6a93654eef
Oct 30 21:36:58.509271 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.509233    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-grk5c"
Oct 30 21:36:58.519494 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.518290    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ggqpf"
Oct 30 21:36:58.519898 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:58.519873    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod948affe7_aaf8_4d3e_a515_ed9552d19d90.slice/crio-0cc39ae0c0738db00bdba883f5a07a93a1a8a36e787b9a29a2566f92369b68da WatchSource:0}: Error finding container 0cc39ae0c0738db00bdba883f5a07a93a1a8a36e787b9a29a2566f92369b68da: Status 404 returned error can't find the container with id 0cc39ae0c0738db00bdba883f5a07a93a1a8a36e787b9a29a2566f92369b68da
Oct 30 21:36:58.521589 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.521570    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:36:58.523984 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.523962    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dbpfd"
Oct 30 21:36:58.527472 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:58.527446    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod726700d7_c252_479e_abdd_18c0573102d1.slice/crio-ea38fb4da192cb36f586fb61f29453437cad03605c6d13c34528d69f83ad72cf WatchSource:0}: Error finding container ea38fb4da192cb36f586fb61f29453437cad03605c6d13c34528d69f83ad72cf: Status 404 returned error can't find the container with id ea38fb4da192cb36f586fb61f29453437cad03605c6d13c34528d69f83ad72cf
Oct 30 21:36:58.528231 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.528213    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4vb42"
Oct 30 21:36:58.530852 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:58.530831    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2360fddc_52a5_42a5_82ca_a05aff84d217.slice/crio-e1686a9277d1b295a6c32714b59c5d06fb328bad430c3643db035b87b78a6dc1 WatchSource:0}: Error finding container e1686a9277d1b295a6c32714b59c5d06fb328bad430c3643db035b87b78a6dc1: Status 404 returned error can't find the container with id e1686a9277d1b295a6c32714b59c5d06fb328bad430c3643db035b87b78a6dc1
Oct 30 21:36:58.532118 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.532102    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4frz9"
Oct 30 21:36:58.532681 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:58.532663    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod909906d5_5049_4742_9b5a_b9c03ed5cc39.slice/crio-4fa38f1049cb90c4ec0a82e2f000d049e39133939c55cbc3f898a79fae258b78 WatchSource:0}: Error finding container 4fa38f1049cb90c4ec0a82e2f000d049e39133939c55cbc3f898a79fae258b78: Status 404 returned error can't find the container with id 4fa38f1049cb90c4ec0a82e2f000d049e39133939c55cbc3f898a79fae258b78
Oct 30 21:36:58.538461 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:58.538443    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d9d43c2_40da_47a8_adb4_d3ba378d77d9.slice/crio-eec01d450a8b96b3679b58621236996f1dbf1f8666a73877528cca3ecb2abcce WatchSource:0}: Error finding container eec01d450a8b96b3679b58621236996f1dbf1f8666a73877528cca3ecb2abcce: Status 404 returned error can't find the container with id eec01d450a8b96b3679b58621236996f1dbf1f8666a73877528cca3ecb2abcce
Oct 30 21:36:58.541552 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:36:58.541532    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb8bde01_2bc6_4301_aa2f_2858158bd54e.slice/crio-0c8fda6bdba774f960398c99876ee2b975ed7571ccb35945e3d16ccbe83646b9 WatchSource:0}: Error finding container 0c8fda6bdba774f960398c99876ee2b975ed7571ccb35945e3d16ccbe83646b9: Status 404 returned error can't find the container with id 0c8fda6bdba774f960398c99876ee2b975ed7571ccb35945e3d16ccbe83646b9
Oct 30 21:36:58.810184 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.810102    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-metrics-certs\") pod \"network-metrics-daemon-l9kxl\" (UID: \"7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2\") " pod="openshift-multus/network-metrics-daemon-l9kxl"
Oct 30 21:36:58.810184 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.810136    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e615be3-41d7-4f82-90d7-7ce71ed9190b-original-pull-secret\") pod \"global-pull-secret-syncer-z9h2r\" (UID: \"6e615be3-41d7-4f82-90d7-7ce71ed9190b\") " pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:36:58.810366 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:58.810252    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:36:58.810366 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:58.810257    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:36:58.810366 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:58.810306    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e615be3-41d7-4f82-90d7-7ce71ed9190b-original-pull-secret podName:6e615be3-41d7-4f82-90d7-7ce71ed9190b nodeName:}" failed. No retries permitted until 2026-10-30 21:36:59.810290909 +0000 UTC m=+4.136783442 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e615be3-41d7-4f82-90d7-7ce71ed9190b-original-pull-secret") pod "global-pull-secret-syncer-z9h2r" (UID: "6e615be3-41d7-4f82-90d7-7ce71ed9190b") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:36:58.810366 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:58.810321    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-metrics-certs podName:7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2 nodeName:}" failed. No retries permitted until 2026-10-30 21:36:59.810314546 +0000 UTC m=+4.136807077 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-metrics-certs") pod "network-metrics-daemon-l9kxl" (UID: "7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:36:58.910901 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:58.910865    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxmv7\" (UniqueName: \"kubernetes.io/projected/9eefa84c-5304-4c2a-89cf-3304097ecf42-kube-api-access-gxmv7\") pod \"network-check-target-wpc6f\" (UID: \"9eefa84c-5304-4c2a-89cf-3304097ecf42\") " pod="openshift-network-diagnostics/network-check-target-wpc6f"
Oct 30 21:36:58.911053 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:58.910987    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:36:58.911053 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:58.911003    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:36:58.911053 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:58.911012    2568 projected.go:196] Error preparing data for projected volume kube-api-access-gxmv7 for pod openshift-network-diagnostics/network-check-target-wpc6f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:36:58.911140 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:58.911064    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9eefa84c-5304-4c2a-89cf-3304097ecf42-kube-api-access-gxmv7 podName:9eefa84c-5304-4c2a-89cf-3304097ecf42 nodeName:}" failed. No retries permitted until 2026-10-30 21:36:59.911047923 +0000 UTC m=+4.237540456 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-gxmv7" (UniqueName: "kubernetes.io/projected/9eefa84c-5304-4c2a-89cf-3304097ecf42-kube-api-access-gxmv7") pod "network-check-target-wpc6f" (UID: "9eefa84c-5304-4c2a-89cf-3304097ecf42") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:36:59.217155 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:59.217065    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9kxl"
Oct 30 21:36:59.217693 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:59.217229    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9kxl" podUID="7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2"
Oct 30 21:36:59.225344 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:59.225317    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4frz9" event={"ID":"cb8bde01-2bc6-4301-aa2f-2858158bd54e","Type":"ContainerStarted","Data":"0c8fda6bdba774f960398c99876ee2b975ed7571ccb35945e3d16ccbe83646b9"}
Oct 30 21:36:59.227226 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:59.227193    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wzntd" event={"ID":"2360fddc-52a5-42a5-82ca-a05aff84d217","Type":"ContainerStarted","Data":"e1686a9277d1b295a6c32714b59c5d06fb328bad430c3643db035b87b78a6dc1"}
Oct 30 21:36:59.228586 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:59.228556    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ggqpf" event={"ID":"726700d7-c252-479e-abdd-18c0573102d1","Type":"ContainerStarted","Data":"ea38fb4da192cb36f586fb61f29453437cad03605c6d13c34528d69f83ad72cf"}
Oct 30 21:36:59.230358 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:59.230325    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2wb5p" event={"ID":"07b45ded-5ba8-49dc-b6ab-c170ca47af9b","Type":"ContainerStarted","Data":"9d70bbf91a12f4450deb83490604353f3607520ccdbe175a3ecc4b6a93654eef"}
Oct 30 21:36:59.232129 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:59.232103    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-f9gtx" event={"ID":"457fd859-0880-43f8-b712-2fd90c9debb0","Type":"ContainerStarted","Data":"8cbeffb016005bf058c1408477fa97d052e5e650265e29ebc098011a6888d1e8"}
Oct 30 21:36:59.233520 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:59.233493    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4vb42" event={"ID":"6d9d43c2-40da-47a8-adb4-d3ba378d77d9","Type":"ContainerStarted","Data":"eec01d450a8b96b3679b58621236996f1dbf1f8666a73877528cca3ecb2abcce"}
Oct 30 21:36:59.234810 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:59.234786    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dbpfd" event={"ID":"909906d5-5049-4742-9b5a-b9c03ed5cc39","Type":"ContainerStarted","Data":"4fa38f1049cb90c4ec0a82e2f000d049e39133939c55cbc3f898a79fae258b78"}
Oct 30 21:36:59.236748 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:59.236714    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-grk5c" event={"ID":"948affe7-aaf8-4d3e-a515-ed9552d19d90","Type":"ContainerStarted","Data":"0cc39ae0c0738db00bdba883f5a07a93a1a8a36e787b9a29a2566f92369b68da"}
Oct 30 21:36:59.238453 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:59.238403    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9" event={"ID":"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2","Type":"ContainerStarted","Data":"c4c22dc33e25512d0c3ae645987661746b0c1ac84cf568e3250994918ee72d31"}
Oct 30 21:36:59.289853 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:59.289770    2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 21:31:58 +0000 UTC" deadline="2028-07-03 20:21:58.16931532 +0000 UTC"
Oct 30 21:36:59.289853 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:59.289850    2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14686h44m58.879470519s"
Oct 30 21:36:59.819538 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:59.819497    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-metrics-certs\") pod \"network-metrics-daemon-l9kxl\" (UID: \"7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2\") " pod="openshift-multus/network-metrics-daemon-l9kxl"
Oct 30 21:36:59.819732 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:59.819556    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e615be3-41d7-4f82-90d7-7ce71ed9190b-original-pull-secret\") pod \"global-pull-secret-syncer-z9h2r\" (UID: \"6e615be3-41d7-4f82-90d7-7ce71ed9190b\") " pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:36:59.819732 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:59.819723    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:36:59.819840 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:59.819787    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e615be3-41d7-4f82-90d7-7ce71ed9190b-original-pull-secret podName:6e615be3-41d7-4f82-90d7-7ce71ed9190b nodeName:}" failed. No retries permitted until 2026-10-30 21:37:01.819769756 +0000 UTC m=+6.146262298 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e615be3-41d7-4f82-90d7-7ce71ed9190b-original-pull-secret") pod "global-pull-secret-syncer-z9h2r" (UID: "6e615be3-41d7-4f82-90d7-7ce71ed9190b") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:36:59.819903 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:59.819851    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:36:59.819903 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:59.819885    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-metrics-certs podName:7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:01.819873312 +0000 UTC m=+6.146365863 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-metrics-certs") pod "network-metrics-daemon-l9kxl" (UID: "7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:36:59.920177 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:36:59.920127    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxmv7\" (UniqueName: \"kubernetes.io/projected/9eefa84c-5304-4c2a-89cf-3304097ecf42-kube-api-access-gxmv7\") pod \"network-check-target-wpc6f\" (UID: \"9eefa84c-5304-4c2a-89cf-3304097ecf42\") " pod="openshift-network-diagnostics/network-check-target-wpc6f"
Oct 30 21:36:59.920391 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:59.920374    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:36:59.920481 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:59.920402    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:36:59.920481 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:59.920416    2568 projected.go:196] Error preparing data for projected volume kube-api-access-gxmv7 for pod openshift-network-diagnostics/network-check-target-wpc6f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:36:59.920481 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:36:59.920478    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9eefa84c-5304-4c2a-89cf-3304097ecf42-kube-api-access-gxmv7 podName:9eefa84c-5304-4c2a-89cf-3304097ecf42 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:01.920459134 +0000 UTC m=+6.246951668 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-gxmv7" (UniqueName: "kubernetes.io/projected/9eefa84c-5304-4c2a-89cf-3304097ecf42-kube-api-access-gxmv7") pod "network-check-target-wpc6f" (UID: "9eefa84c-5304-4c2a-89cf-3304097ecf42") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:37:00.219097 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:00.219014    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:37:00.219558 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:00.219159    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9h2r" podUID="6e615be3-41d7-4f82-90d7-7ce71ed9190b"
Oct 30 21:37:00.219558 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:00.219261    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wpc6f"
Oct 30 21:37:00.219558 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:00.219334    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wpc6f" podUID="9eefa84c-5304-4c2a-89cf-3304097ecf42"
Oct 30 21:37:00.290484 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:00.290413    2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 21:31:58 +0000 UTC" deadline="2028-04-07 02:56:18.215797894 +0000 UTC"
Oct 30 21:37:00.290484 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:00.290477    2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12581h19m17.925324977s"
Oct 30 21:37:01.217406 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:01.217201    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9kxl"
Oct 30 21:37:01.217406 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:01.217344    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9kxl" podUID="7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2"
Oct 30 21:37:01.836622 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:01.836498    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-metrics-certs\") pod \"network-metrics-daemon-l9kxl\" (UID: \"7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2\") " pod="openshift-multus/network-metrics-daemon-l9kxl"
Oct 30 21:37:01.837193 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:01.836646    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e615be3-41d7-4f82-90d7-7ce71ed9190b-original-pull-secret\") pod \"global-pull-secret-syncer-z9h2r\" (UID: \"6e615be3-41d7-4f82-90d7-7ce71ed9190b\") " pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:37:01.837193 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:01.836653    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:37:01.837193 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:01.836712    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-metrics-certs podName:7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:05.836696016 +0000 UTC m=+10.163188562 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-metrics-certs") pod "network-metrics-daemon-l9kxl" (UID: "7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:37:01.837193 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:01.836783    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:37:01.837193 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:01.836830    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e615be3-41d7-4f82-90d7-7ce71ed9190b-original-pull-secret podName:6e615be3-41d7-4f82-90d7-7ce71ed9190b nodeName:}" failed. No retries permitted until 2026-10-30 21:37:05.836818421 +0000 UTC m=+10.163310957 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e615be3-41d7-4f82-90d7-7ce71ed9190b-original-pull-secret") pod "global-pull-secret-syncer-z9h2r" (UID: "6e615be3-41d7-4f82-90d7-7ce71ed9190b") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:37:01.937160 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:01.936970    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxmv7\" (UniqueName: \"kubernetes.io/projected/9eefa84c-5304-4c2a-89cf-3304097ecf42-kube-api-access-gxmv7\") pod \"network-check-target-wpc6f\" (UID: \"9eefa84c-5304-4c2a-89cf-3304097ecf42\") " pod="openshift-network-diagnostics/network-check-target-wpc6f"
Oct 30 21:37:01.937336 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:01.937322    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:37:01.937406 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:01.937345    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:37:01.937458 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:01.937411    2568 projected.go:196] Error preparing data for projected volume kube-api-access-gxmv7 for pod openshift-network-diagnostics/network-check-target-wpc6f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:37:01.937506 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:01.937465    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9eefa84c-5304-4c2a-89cf-3304097ecf42-kube-api-access-gxmv7 podName:9eefa84c-5304-4c2a-89cf-3304097ecf42 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:05.937446872 +0000 UTC m=+10.263939419 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-gxmv7" (UniqueName: "kubernetes.io/projected/9eefa84c-5304-4c2a-89cf-3304097ecf42-kube-api-access-gxmv7") pod "network-check-target-wpc6f" (UID: "9eefa84c-5304-4c2a-89cf-3304097ecf42") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:37:02.219747 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:02.219360    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wpc6f"
Oct 30 21:37:02.219747 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:02.219488    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wpc6f" podUID="9eefa84c-5304-4c2a-89cf-3304097ecf42"
Oct 30 21:37:02.219747 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:02.219552    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:37:02.219747 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:02.219614    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9h2r" podUID="6e615be3-41d7-4f82-90d7-7ce71ed9190b"
Oct 30 21:37:02.273519 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:02.273474    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-46.us-west-2.compute.internal" event={"ID":"1d836aa5280c51c365d2d598465f9d5c","Type":"ContainerStarted","Data":"a3489bcd07eaac036a281d6878c4e8be7fab01b45fee9df3bbb4943394fb9627"}
Oct 30 21:37:02.278663 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:02.278626    2568 generic.go:356] "Generic (PLEG): container finished" podID="85a4f26b44393e519ecc5f1c6d0f8225" containerID="26ced90e504e27dc1a49d45e3c1b57df3ccfef918fd6108829246632096ba5d0" exitCode=0
Oct 30 21:37:02.278827 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:02.278688    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal" event={"ID":"85a4f26b44393e519ecc5f1c6d0f8225","Type":"ContainerDied","Data":"26ced90e504e27dc1a49d45e3c1b57df3ccfef918fd6108829246632096ba5d0"}
Oct 30 21:37:02.495102 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:02.495046    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-46.us-west-2.compute.internal" podStartSLOduration=5.495030838 podStartE2EDuration="5.495030838s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:36:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:02.386295406 +0000 UTC m=+6.712787965" watchObservedRunningTime="2026-10-30 21:37:02.495030838 +0000 UTC m=+6.821523408"
Oct 30 21:37:03.216586 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:03.216547    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9kxl"
Oct 30 21:37:03.217053 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:03.216688    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9kxl" podUID="7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2"
Oct 30 21:37:03.287126 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:03.287084    2568 generic.go:356] "Generic (PLEG): container finished" podID="cb8bde01-2bc6-4301-aa2f-2858158bd54e" containerID="66659b05ba33b40eb5c5627af97ed2d6430bbcc4aa97e469eae5e44d2774c2e7" exitCode=0
Oct 30 21:37:03.287323 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:03.287196    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4frz9" event={"ID":"cb8bde01-2bc6-4301-aa2f-2858158bd54e","Type":"ContainerDied","Data":"66659b05ba33b40eb5c5627af97ed2d6430bbcc4aa97e469eae5e44d2774c2e7"}
Oct 30 21:37:03.292712 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:03.292676    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal" event={"ID":"85a4f26b44393e519ecc5f1c6d0f8225","Type":"ContainerStarted","Data":"8ba5c0a58e456e5c683dab20b394e024d17ef3f4d3b3f0c0c216fc9d55ada8c3"}
Oct 30 21:37:03.513718 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:03.513615    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal" podStartSLOduration=6.513594641 podStartE2EDuration="6.513594641s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:36:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:03.513575413 +0000 UTC m=+7.840067965" watchObservedRunningTime="2026-10-30 21:37:03.513594641 +0000 UTC m=+7.840087195"
Oct 30 21:37:04.218998 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:04.218965    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:37:04.219497 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:04.219106    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9h2r" podUID="6e615be3-41d7-4f82-90d7-7ce71ed9190b"
Oct 30 21:37:04.219556 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:04.219508    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wpc6f"
Oct 30 21:37:04.219610 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:04.219597    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wpc6f" podUID="9eefa84c-5304-4c2a-89cf-3304097ecf42"
Oct 30 21:37:04.298126 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:04.298095    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4vb42" event={"ID":"6d9d43c2-40da-47a8-adb4-d3ba378d77d9","Type":"ContainerStarted","Data":"6be45dd871efdb26e77853a531ccdbfbff3b22278686629f9bba5ea3f5c97c16"}
Oct 30 21:37:05.217097 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:05.217069    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9kxl"
Oct 30 21:37:05.217267 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:05.217229    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9kxl" podUID="7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2"
Oct 30 21:37:05.304888 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:05.304144    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-grk5c" event={"ID":"948affe7-aaf8-4d3e-a515-ed9552d19d90","Type":"ContainerStarted","Data":"ceb4d4310bbe85982a2e1d359ebfc5d9d58c1dfe26efaff915101c3cea85c855"}
Oct 30 21:37:05.308836 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:05.308541    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9" event={"ID":"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2","Type":"ContainerStarted","Data":"9cf0ee895c1831605470006862e02b1073a2629aa201823cf474978a59659f7a"}
Oct 30 21:37:05.414875 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:05.413881    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-grk5c" podStartSLOduration=3.793526275 podStartE2EDuration="9.413864518s" totalImagesPullingTime="5.620338243s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:36:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:05.413510177 +0000 UTC m=+9.740002732" watchObservedRunningTime="2026-10-30 21:37:05.413864518 +0000 UTC m=+9.740357072"
Oct 30 21:37:05.414875 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:05.413986    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-4vb42" podStartSLOduration=4.566478946 podStartE2EDuration="9.413975917s" totalImagesPullingTime="4.847496971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:36:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:04.407251546 +0000 UTC m=+8.733744097" watchObservedRunningTime="2026-10-30 21:37:05.413975917 +0000 UTC m=+9.740468473"
Oct 30 21:37:05.874284 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:05.874177    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-metrics-certs\") pod \"network-metrics-daemon-l9kxl\" (UID: \"7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2\") " pod="openshift-multus/network-metrics-daemon-l9kxl"
Oct 30 21:37:05.874284 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:05.874226    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e615be3-41d7-4f82-90d7-7ce71ed9190b-original-pull-secret\") pod \"global-pull-secret-syncer-z9h2r\" (UID: \"6e615be3-41d7-4f82-90d7-7ce71ed9190b\") " pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:37:05.874508 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:05.874365    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:37:05.874508 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:05.874428    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e615be3-41d7-4f82-90d7-7ce71ed9190b-original-pull-secret podName:6e615be3-41d7-4f82-90d7-7ce71ed9190b nodeName:}" failed. No retries permitted until 2026-10-30 21:37:13.874411234 +0000 UTC m=+18.200903766 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e615be3-41d7-4f82-90d7-7ce71ed9190b-original-pull-secret") pod "global-pull-secret-syncer-z9h2r" (UID: "6e615be3-41d7-4f82-90d7-7ce71ed9190b") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:37:05.875075 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:05.874763    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:37:05.875075 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:05.874819    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-metrics-certs podName:7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:13.874803994 +0000 UTC m=+18.201296528 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-metrics-certs") pod "network-metrics-daemon-l9kxl" (UID: "7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:37:05.975356 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:05.974976    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxmv7\" (UniqueName: \"kubernetes.io/projected/9eefa84c-5304-4c2a-89cf-3304097ecf42-kube-api-access-gxmv7\") pod \"network-check-target-wpc6f\" (UID: \"9eefa84c-5304-4c2a-89cf-3304097ecf42\") " pod="openshift-network-diagnostics/network-check-target-wpc6f"
Oct 30 21:37:05.975356 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:05.975166    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:37:05.975356 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:05.975185    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:37:05.975356 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:05.975199    2568 projected.go:196] Error preparing data for projected volume kube-api-access-gxmv7 for pod openshift-network-diagnostics/network-check-target-wpc6f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:37:05.975356 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:05.975270    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9eefa84c-5304-4c2a-89cf-3304097ecf42-kube-api-access-gxmv7 podName:9eefa84c-5304-4c2a-89cf-3304097ecf42 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:13.975250936 +0000 UTC m=+18.301743485 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-gxmv7" (UniqueName: "kubernetes.io/projected/9eefa84c-5304-4c2a-89cf-3304097ecf42-kube-api-access-gxmv7") pod "network-check-target-wpc6f" (UID: "9eefa84c-5304-4c2a-89cf-3304097ecf42") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:37:06.218449 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:06.218371    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wpc6f"
Oct 30 21:37:06.218617 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:06.218371    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:37:06.218617 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:06.218510    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wpc6f" podUID="9eefa84c-5304-4c2a-89cf-3304097ecf42"
Oct 30 21:37:06.218737 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:06.218609    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9h2r" podUID="6e615be3-41d7-4f82-90d7-7ce71ed9190b"
Oct 30 21:37:06.893072 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:06.888716    2568 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-4vb42"
Oct 30 21:37:06.893072 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:06.890384    2568 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-4vb42"
Oct 30 21:37:07.217319 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:07.217103    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9kxl"
Oct 30 21:37:07.217319 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:07.217202    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9kxl" podUID="7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2"
Oct 30 21:37:07.315001 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:07.314957    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dbpfd" event={"ID":"909906d5-5049-4742-9b5a-b9c03ed5cc39","Type":"ContainerStarted","Data":"6c1c7d69b10c1d5d410b23558dcb76e95ac724da8a68fcbd542d547efa357b0e"}
Oct 30 21:37:07.319114 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:07.318145    2568 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 21:37:07.319114 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:07.319051    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ggqpf" event={"ID":"726700d7-c252-479e-abdd-18c0573102d1","Type":"ContainerStarted","Data":"88a3e77401f23b3c48c6c66ae7556bc9cb11c22892de5102fe3c9ac2b01bcab6"}
Oct 30 21:37:07.525171 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:07.525144    2568 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 21:37:07.538197 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:07.538084    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-ggqpf" podStartSLOduration=3.452476882 podStartE2EDuration="11.538067767s" totalImagesPullingTime="8.085590885s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:36:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:07.536386797 +0000 UTC m=+11.862879354" watchObservedRunningTime="2026-10-30 21:37:07.538067767 +0000 UTC m=+11.864560320"
Oct 30 21:37:07.541831 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:07.541771    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-dbpfd" podStartSLOduration=3.238386519 podStartE2EDuration="11.54175274s" totalImagesPullingTime="8.303366221s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:36:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:07.425171137 +0000 UTC m=+11.751663691" watchObservedRunningTime="2026-10-30 21:37:07.54175274 +0000 UTC m=+11.868245295"
Oct 30 21:37:08.174063 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:08.173808    2568 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T21:37:07.525166695Z","UUID":"903487d4-4a91-44db-8324-4885b8b15eff","Handler":null,"Name":"","Endpoint":""}
Oct 30 21:37:08.175329 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:08.175306    2568 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 21:37:08.175329 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:08.175332    2568 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 21:37:08.218326 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:08.216740    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:37:08.218326 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:08.216884    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9h2r" podUID="6e615be3-41d7-4f82-90d7-7ce71ed9190b"
Oct 30 21:37:08.218326 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:08.216902    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wpc6f"
Oct 30 21:37:08.218326 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:08.217014    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wpc6f" podUID="9eefa84c-5304-4c2a-89cf-3304097ecf42"
Oct 30 21:37:08.323309 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:08.323271    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9" event={"ID":"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2","Type":"ContainerStarted","Data":"f3164874a009deb895b83d8c7d54a6f758b941a720077136c49af79949636c76"}
Oct 30 21:37:08.325443 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:08.325407    2568 generic.go:356] "Generic (PLEG): container finished" podID="cb8bde01-2bc6-4301-aa2f-2858158bd54e" containerID="bb52b4c2ed8745a45397007c68e36cfeb009c46a29227600601612fb4d14dc1c" exitCode=0
Oct 30 21:37:08.325568 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:08.325450    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4frz9" event={"ID":"cb8bde01-2bc6-4301-aa2f-2858158bd54e","Type":"ContainerDied","Data":"bb52b4c2ed8745a45397007c68e36cfeb009c46a29227600601612fb4d14dc1c"}
Oct 30 21:37:09.216672 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:09.216637    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9kxl"
Oct 30 21:37:09.217187 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:09.216762    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9kxl" podUID="7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2"
Oct 30 21:37:09.329344 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:09.328943    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2wb5p" event={"ID":"07b45ded-5ba8-49dc-b6ab-c170ca47af9b","Type":"ContainerStarted","Data":"39e44fb449a1a80adaededfc07a426697c0a28b4bc924bea75cbcda8b24b7ef9"}
Oct 30 21:37:09.331895 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:09.331866    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-f9gtx" event={"ID":"457fd859-0880-43f8-b712-2fd90c9debb0","Type":"ContainerStarted","Data":"684f4fe2af0e387ba65ec75424cbc68729ef417b0be1f673269eddce151a204a"}
Oct 30 21:37:09.550192 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:09.550129    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-2wb5p" podStartSLOduration=4.612332932 podStartE2EDuration="13.550109835s" totalImagesPullingTime="8.937776903s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:36:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:09.439136177 +0000 UTC m=+13.765628731" watchObservedRunningTime="2026-10-30 21:37:09.550109835 +0000 UTC m=+13.876602391"
Oct 30 21:37:09.550551 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:09.550521    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-f9gtx" podStartSLOduration=3.053912158 podStartE2EDuration="13.550513508s" totalImagesPullingTime="10.49660135s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:36:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:09.549967476 +0000 UTC m=+13.876460032" watchObservedRunningTime="2026-10-30 21:37:09.550513508 +0000 UTC m=+13.877006063"
Oct 30 21:37:10.216885 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:10.216788    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wpc6f"
Oct 30 21:37:10.217521 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:10.216922    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wpc6f" podUID="9eefa84c-5304-4c2a-89cf-3304097ecf42"
Oct 30 21:37:10.217521 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:10.217373    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:37:10.217521 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:10.217481    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9h2r" podUID="6e615be3-41d7-4f82-90d7-7ce71ed9190b"
Oct 30 21:37:10.336403 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:10.336366    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9" event={"ID":"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2","Type":"ContainerStarted","Data":"75478021dbfd34dd0b688bfea372d9aa92f3cff61a2d3e1e085d3ccbf15dd33e"}
Oct 30 21:37:10.338882 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:10.338855    2568 generic.go:356] "Generic (PLEG): container finished" podID="cb8bde01-2bc6-4301-aa2f-2858158bd54e" containerID="3eef80e415e775a8594f1d1354606ac0b273be2df4d70aa16ba324292d2a1a67" exitCode=0
Oct 30 21:37:10.339024 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:10.338946    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4frz9" event={"ID":"cb8bde01-2bc6-4301-aa2f-2858158bd54e","Type":"ContainerDied","Data":"3eef80e415e775a8594f1d1354606ac0b273be2df4d70aa16ba324292d2a1a67"}
Oct 30 21:37:10.448213 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:10.448157    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9" podStartSLOduration=3.965055785 podStartE2EDuration="14.44814027s" totalImagesPullingTime="10.483084485s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:36:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:10.447642751 +0000 UTC m=+14.774135305" watchObservedRunningTime="2026-10-30 21:37:10.44814027 +0000 UTC m=+14.774632825"
Oct 30 21:37:11.216861 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:11.216830    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9kxl"
Oct 30 21:37:11.217045 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:11.216972    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9kxl" podUID="7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2"
Oct 30 21:37:12.217321 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:12.217057    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:37:12.217927 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:12.217064    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wpc6f"
Oct 30 21:37:12.217927 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:12.217378    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9h2r" podUID="6e615be3-41d7-4f82-90d7-7ce71ed9190b"
Oct 30 21:37:12.217927 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:12.217450    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wpc6f" podUID="9eefa84c-5304-4c2a-89cf-3304097ecf42"
Oct 30 21:37:12.346843 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:12.346813    2568 generic.go:356] "Generic (PLEG): container finished" podID="cb8bde01-2bc6-4301-aa2f-2858158bd54e" containerID="e2ba3ba6ed4014ad938f8b91b8c7d0dc641be61bc040b2ee1d40e0eb6dd965e2" exitCode=0
Oct 30 21:37:12.347021 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:12.346863    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4frz9" event={"ID":"cb8bde01-2bc6-4301-aa2f-2858158bd54e","Type":"ContainerDied","Data":"e2ba3ba6ed4014ad938f8b91b8c7d0dc641be61bc040b2ee1d40e0eb6dd965e2"}
Oct 30 21:37:13.216415 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:13.216389    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9kxl"
Oct 30 21:37:13.216551 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:13.216491    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9kxl" podUID="7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2"
Oct 30 21:37:13.939070 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:13.938877    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-metrics-certs\") pod \"network-metrics-daemon-l9kxl\" (UID: \"7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2\") " pod="openshift-multus/network-metrics-daemon-l9kxl"
Oct 30 21:37:13.939520 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:13.939093    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e615be3-41d7-4f82-90d7-7ce71ed9190b-original-pull-secret\") pod \"global-pull-secret-syncer-z9h2r\" (UID: \"6e615be3-41d7-4f82-90d7-7ce71ed9190b\") " pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:37:13.939520 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:13.939033    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:37:13.939520 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:13.939221    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:37:13.939520 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:13.939271    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-metrics-certs podName:7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:29.939251906 +0000 UTC m=+34.265744455 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-metrics-certs") pod "network-metrics-daemon-l9kxl" (UID: "7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:37:13.939520 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:13.939317    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e615be3-41d7-4f82-90d7-7ce71ed9190b-original-pull-secret podName:6e615be3-41d7-4f82-90d7-7ce71ed9190b nodeName:}" failed. No retries permitted until 2026-10-30 21:37:29.939309624 +0000 UTC m=+34.265802155 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e615be3-41d7-4f82-90d7-7ce71ed9190b-original-pull-secret") pod "global-pull-secret-syncer-z9h2r" (UID: "6e615be3-41d7-4f82-90d7-7ce71ed9190b") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:37:14.040167 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:14.040095    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxmv7\" (UniqueName: \"kubernetes.io/projected/9eefa84c-5304-4c2a-89cf-3304097ecf42-kube-api-access-gxmv7\") pod \"network-check-target-wpc6f\" (UID: \"9eefa84c-5304-4c2a-89cf-3304097ecf42\") " pod="openshift-network-diagnostics/network-check-target-wpc6f"
Oct 30 21:37:14.040314 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:14.040262    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:37:14.040314 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:14.040282    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:37:14.040314 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:14.040292    2568 projected.go:196] Error preparing data for projected volume kube-api-access-gxmv7 for pod openshift-network-diagnostics/network-check-target-wpc6f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:37:14.040424 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:14.040345    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9eefa84c-5304-4c2a-89cf-3304097ecf42-kube-api-access-gxmv7 podName:9eefa84c-5304-4c2a-89cf-3304097ecf42 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:30.040330297 +0000 UTC m=+34.366822845 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-gxmv7" (UniqueName: "kubernetes.io/projected/9eefa84c-5304-4c2a-89cf-3304097ecf42-kube-api-access-gxmv7") pod "network-check-target-wpc6f" (UID: "9eefa84c-5304-4c2a-89cf-3304097ecf42") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:37:14.217168 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:14.217132    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wpc6f"
Oct 30 21:37:14.217350 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:14.217288    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wpc6f" podUID="9eefa84c-5304-4c2a-89cf-3304097ecf42"
Oct 30 21:37:14.217350 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:14.217325    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:37:14.217439 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:14.217404    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9h2r" podUID="6e615be3-41d7-4f82-90d7-7ce71ed9190b"
Oct 30 21:37:14.354549 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:14.354468    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzntd_2360fddc-52a5-42a5-82ca-a05aff84d217/ovn-acl-logging/0.log"
Oct 30 21:37:14.354889 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:14.354860    2568 generic.go:356] "Generic (PLEG): container finished" podID="2360fddc-52a5-42a5-82ca-a05aff84d217" containerID="327a6e103e9b22f0e47ff17afc2a9a0f00bfd68b28b5cf9ce0dac0a2449d99c2" exitCode=1
Oct 30 21:37:14.354983 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:14.354908    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wzntd" event={"ID":"2360fddc-52a5-42a5-82ca-a05aff84d217","Type":"ContainerStarted","Data":"6ddefe4c16335042a2dee478e4fdea123890a4a0bc7b5fc3ff72ba618c7843d1"}
Oct 30 21:37:14.354983 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:14.354968    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wzntd" event={"ID":"2360fddc-52a5-42a5-82ca-a05aff84d217","Type":"ContainerStarted","Data":"34f76816eba4b1c3b7ae66997ff4922c24d9215cb2eff4448e4b1e5013a68ed8"}
Oct 30 21:37:14.355060 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:14.355006    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wzntd" event={"ID":"2360fddc-52a5-42a5-82ca-a05aff84d217","Type":"ContainerStarted","Data":"e7bfdf95ea520184030d63cfe66f7a8012f965c890703b33a5178dad1664088d"}
Oct 30 21:37:14.355060 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:14.355041    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wzntd" event={"ID":"2360fddc-52a5-42a5-82ca-a05aff84d217","Type":"ContainerStarted","Data":"eb06c08eb4d8d3aaba4a30df257e340975d23def1c03cddf0f7c15772e52a626"}
Oct 30 21:37:14.355144 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:14.355082    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wzntd" event={"ID":"2360fddc-52a5-42a5-82ca-a05aff84d217","Type":"ContainerDied","Data":"327a6e103e9b22f0e47ff17afc2a9a0f00bfd68b28b5cf9ce0dac0a2449d99c2"}
Oct 30 21:37:14.355144 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:14.355121    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wzntd" event={"ID":"2360fddc-52a5-42a5-82ca-a05aff84d217","Type":"ContainerStarted","Data":"1daf68f03b55500d70df8b92f48688a066af1d1e7cb183bae9c74c540d9a3c0c"}
Oct 30 21:37:15.217137 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:15.217101    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9kxl"
Oct 30 21:37:15.217687 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:15.217209    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9kxl" podUID="7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2"
Oct 30 21:37:15.312404 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:15.312367    2568 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-4vb42"
Oct 30 21:37:15.312536 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:15.312520    2568 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 21:37:15.312978 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:15.312959    2568 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-4vb42"
Oct 30 21:37:15.359183 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:15.359153    2568 generic.go:356] "Generic (PLEG): container finished" podID="cb8bde01-2bc6-4301-aa2f-2858158bd54e" containerID="aac1cec8a74591c97a22b7944c1b5192d7ba619ffe3dbcb3fe6f44171b0b5ed6" exitCode=0
Oct 30 21:37:15.359322 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:15.359248    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4frz9" event={"ID":"cb8bde01-2bc6-4301-aa2f-2858158bd54e","Type":"ContainerDied","Data":"aac1cec8a74591c97a22b7944c1b5192d7ba619ffe3dbcb3fe6f44171b0b5ed6"}
Oct 30 21:37:16.217206 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:16.217128    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:37:16.217695 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:16.217256    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wpc6f"
Oct 30 21:37:16.217695 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:16.217259    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9h2r" podUID="6e615be3-41d7-4f82-90d7-7ce71ed9190b"
Oct 30 21:37:16.217695 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:16.217316    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wpc6f" podUID="9eefa84c-5304-4c2a-89cf-3304097ecf42"
Oct 30 21:37:16.364537 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:16.364506    2568 generic.go:356] "Generic (PLEG): container finished" podID="cb8bde01-2bc6-4301-aa2f-2858158bd54e" containerID="acecb3853fc6c693b108236e9b840bd7464447cf4201b998563860b55c1f275c" exitCode=0
Oct 30 21:37:16.364713 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:16.364591    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4frz9" event={"ID":"cb8bde01-2bc6-4301-aa2f-2858158bd54e","Type":"ContainerDied","Data":"acecb3853fc6c693b108236e9b840bd7464447cf4201b998563860b55c1f275c"}
Oct 30 21:37:16.367641 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:16.367624    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzntd_2360fddc-52a5-42a5-82ca-a05aff84d217/ovn-acl-logging/0.log"
Oct 30 21:37:16.367974 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:16.367952    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wzntd" event={"ID":"2360fddc-52a5-42a5-82ca-a05aff84d217","Type":"ContainerStarted","Data":"ce93096b6595f8ff71fd4cce0fdacf59865fb6255611b811d30e2dbff54c5a1c"}
Oct 30 21:37:17.216928 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:17.216882    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9kxl"
Oct 30 21:37:17.217087 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:17.216996    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9kxl" podUID="7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2"
Oct 30 21:37:17.372851 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:17.372819    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4frz9" event={"ID":"cb8bde01-2bc6-4301-aa2f-2858158bd54e","Type":"ContainerStarted","Data":"555be04e23953905f4266989eb8896ad647b5830b6ccf945f75601689cfda3ca"}
Oct 30 21:37:17.584499 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:17.584398    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-4frz9" podStartSLOduration=8.155954415 podStartE2EDuration="21.584380514s" totalImagesPullingTime="13.428426099s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:36:56 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:17.53749472 +0000 UTC m=+21.863987274" watchObservedRunningTime="2026-10-30 21:37:17.584380514 +0000 UTC m=+21.910873068"
Oct 30 21:37:18.217037 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:18.216998    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:37:18.217037 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:18.217019    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wpc6f"
Oct 30 21:37:18.217261 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:18.217111    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9h2r" podUID="6e615be3-41d7-4f82-90d7-7ce71ed9190b"
Oct 30 21:37:18.217261 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:18.217180    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wpc6f" podUID="9eefa84c-5304-4c2a-89cf-3304097ecf42"
Oct 30 21:37:18.377862 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:18.377758    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzntd_2360fddc-52a5-42a5-82ca-a05aff84d217/ovn-acl-logging/0.log"
Oct 30 21:37:18.378346 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:18.378151    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wzntd" event={"ID":"2360fddc-52a5-42a5-82ca-a05aff84d217","Type":"ContainerStarted","Data":"32862cd7b9c296e8c41495b8ea63d09c178771d0a47d0409172c1d1d8f592f8d"}
Oct 30 21:37:18.378668 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:18.378653    2568 scope.go:122] "RemoveContainer" containerID="327a6e103e9b22f0e47ff17afc2a9a0f00bfd68b28b5cf9ce0dac0a2449d99c2"
Oct 30 21:37:19.216737 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:19.216707    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9kxl"
Oct 30 21:37:19.216894 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:19.216813    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9kxl" podUID="7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2"
Oct 30 21:37:19.383844 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:19.383816    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzntd_2360fddc-52a5-42a5-82ca-a05aff84d217/ovn-acl-logging/0.log"
Oct 30 21:37:19.384437 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:19.384165    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wzntd" event={"ID":"2360fddc-52a5-42a5-82ca-a05aff84d217","Type":"ContainerStarted","Data":"49834ef3c534422779ad20a44b1898508ed2a24e6bfc6fbc60416b272b8c3e9e"}
Oct 30 21:37:19.384437 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:19.384288    2568 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 21:37:19.384509 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:19.384484    2568 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:19.384559 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:19.384549    2568 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:19.400303 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:19.400230    2568 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:19.400451 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:19.400404    2568 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:19.621492 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:19.621396    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-wzntd" podStartSLOduration=8.749239579 podStartE2EDuration="23.62137621s" totalImagesPullingTime="14.872136631s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:36:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:19.599090824 +0000 UTC m=+23.925583377" watchObservedRunningTime="2026-10-30 21:37:19.62137621 +0000 UTC m=+23.947868764"
Oct 30 21:37:20.041444 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:20.041361    2568 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:20.216854 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:20.216821    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wpc6f"
Oct 30 21:37:20.217070 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:20.216867    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:37:20.217070 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:20.216969    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wpc6f" podUID="9eefa84c-5304-4c2a-89cf-3304097ecf42"
Oct 30 21:37:20.217177 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:20.217088    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9h2r" podUID="6e615be3-41d7-4f82-90d7-7ce71ed9190b"
Oct 30 21:37:20.739344 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:20.739263    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-z9h2r"]
Oct 30 21:37:20.739852 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:20.739362    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:37:20.739852 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:20.739444    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9h2r" podUID="6e615be3-41d7-4f82-90d7-7ce71ed9190b"
Oct 30 21:37:20.742424 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:20.742404    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wpc6f"]
Oct 30 21:37:20.742531 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:20.742482    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wpc6f"
Oct 30 21:37:20.742594 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:20.742576    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wpc6f" podUID="9eefa84c-5304-4c2a-89cf-3304097ecf42"
Oct 30 21:37:20.743368 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:20.743344    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-l9kxl"]
Oct 30 21:37:20.743468 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:20.743440    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9kxl"
Oct 30 21:37:20.743549 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:20.743527    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9kxl" podUID="7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2"
Oct 30 21:37:22.216322 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:22.216288    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:37:22.216774 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:22.216291    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9kxl"
Oct 30 21:37:22.216774 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:22.216418    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9h2r" podUID="6e615be3-41d7-4f82-90d7-7ce71ed9190b"
Oct 30 21:37:22.216774 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:22.216515    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9kxl" podUID="7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2"
Oct 30 21:37:22.216774 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:22.216291    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wpc6f"
Oct 30 21:37:22.216774 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:22.216610    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wpc6f" podUID="9eefa84c-5304-4c2a-89cf-3304097ecf42"
Oct 30 21:37:24.217176 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:24.217084    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9kxl"
Oct 30 21:37:24.217614 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:24.217084    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:37:24.217614 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:24.217209    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9kxl" podUID="7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2"
Oct 30 21:37:24.217614 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:24.217316    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9h2r" podUID="6e615be3-41d7-4f82-90d7-7ce71ed9190b"
Oct 30 21:37:24.217614 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:24.217086    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wpc6f"
Oct 30 21:37:24.217614 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:24.217425    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wpc6f" podUID="9eefa84c-5304-4c2a-89cf-3304097ecf42"
Oct 30 21:37:26.217112 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.216941    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:37:26.217613 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.217032    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9kxl"
Oct 30 21:37:26.217613 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:26.217176    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9h2r" podUID="6e615be3-41d7-4f82-90d7-7ce71ed9190b"
Oct 30 21:37:26.217613 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.217060    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wpc6f"
Oct 30 21:37:26.217613 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:26.217315    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9kxl" podUID="7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2"
Oct 30 21:37:26.217613 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:26.217431    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wpc6f" podUID="9eefa84c-5304-4c2a-89cf-3304097ecf42"
Oct 30 21:37:26.417339 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.417265    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeReady"
Oct 30 21:37:26.417484 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.417397    2568 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 21:37:26.539569 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.539534    2568 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-px9dx"]
Oct 30 21:37:26.541959 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.541934    2568 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-jgspb"]
Oct 30 21:37:26.542083 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.542036    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-px9dx"
Oct 30 21:37:26.543468 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.543452    2568 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-nvwdl"]
Oct 30 21:37:26.543595 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.543574    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-jgspb"
Oct 30 21:37:26.544899 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.544884    2568 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-57f669cfc9-d755h"]
Oct 30 21:37:26.545005 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.544991    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-nvwdl"
Oct 30 21:37:26.546270 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.546257    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:26.574008 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.573986    2568 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7d487d746-hf6pr"]
Oct 30 21:37:26.575800 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.575779    2568 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-7c4986d449-pf9px"]
Oct 30 21:37:26.575910 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.575897    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:26.577747 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.577726    2568 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-tlfjh"]
Oct 30 21:37:26.577857 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.577840    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-pf9px"
Oct 30 21:37:26.579435 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.579415    2568 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-qqxdq"]
Oct 30 21:37:26.579540 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.579519    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-tlfjh"
Oct 30 21:37:26.581351 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.581334    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-qqxdq"
Oct 30 21:37:26.586584 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.586515    2568 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"]
Oct 30 21:37:26.591129 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.591106    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Oct 30 21:37:26.591304 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.591182    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Oct 30 21:37:26.591849 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.591830    2568 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-5lpq9"]
Oct 30 21:37:26.592035 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.591948    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:26.592716 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.592699    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Oct 30 21:37:26.592861 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.592846    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:37:26.596258 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.595945    2568 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q2pns"]
Oct 30 21:37:26.597017 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.596999    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-5lpq9"
Oct 30 21:37:26.599112 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.599090    2568 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-wltlg"]
Oct 30 21:37:26.599293 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.599274    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q2pns"
Oct 30 21:37:26.603815 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.603793    2568 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-7lnsg"]
Oct 30 21:37:26.603925 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.603916    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wltlg"
Oct 30 21:37:26.614108 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.614074    2568 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-7tf9f"]
Oct 30 21:37:26.614227 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.614190    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-7lnsg"
Oct 30 21:37:26.617466 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.617439    2568 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-58v4l"]
Oct 30 21:37:26.617566 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.617544    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f"
Oct 30 21:37:26.619579 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.619560    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Oct 30 21:37:26.619951 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.619928    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Oct 30 21:37:26.622277 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.620690    2568 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-6q6pm"]
Oct 30 21:37:26.622277 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.621424    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-58v4l"
Oct 30 21:37:26.624745 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.624723    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-px9dx"]
Oct 30 21:37:26.624848 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.624768    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-nvwdl"]
Oct 30 21:37:26.624848 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.624793    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-jgspb"]
Oct 30 21:37:26.624848 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.624816    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-57f669cfc9-d755h"]
Oct 30 21:37:26.624999 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.624843    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6q6pm"
Oct 30 21:37:26.625412 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.625391    2568 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rqxfc"]
Oct 30 21:37:26.630029 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.630006    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/37186e06-8d56-4c61-ba10-e0408d2e2fa4-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-jgspb\" (UID: \"37186e06-8d56-4c61-ba10-e0408d2e2fa4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-jgspb"
Oct 30 21:37:26.630137 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.630046    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e0535562-96d7-4fc4-a5d5-ad4cc785080e-stats-auth\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:26.630137 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.630128    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/37186e06-8d56-4c61-ba10-e0408d2e2fa4-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-jgspb\" (UID: \"37186e06-8d56-4c61-ba10-e0408d2e2fa4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-jgspb"
Oct 30 21:37:26.630395 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.630167    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e0535562-96d7-4fc4-a5d5-ad4cc785080e-default-certificate\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:26.630395 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.630185    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0535562-96d7-4fc4-a5d5-ad4cc785080e-service-ca-bundle\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:26.630395 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.630213    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phfjb\" (UniqueName: \"kubernetes.io/projected/5f5757a0-0a11-4637-9aa5-4e283792017d-kube-api-access-phfjb\") pod \"kas-connection-checker-685895c7c9-px9dx\" (UID: \"5f5757a0-0a11-4637-9aa5-4e283792017d\") " pod="kube-system/kas-connection-checker-685895c7c9-px9dx"
Oct 30 21:37:26.630395 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.630268    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zn8jz\" (UniqueName: \"kubernetes.io/projected/37186e06-8d56-4c61-ba10-e0408d2e2fa4-kube-api-access-zn8jz\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-jgspb\" (UID: \"37186e06-8d56-4c61-ba10-e0408d2e2fa4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-jgspb"
Oct 30 21:37:26.630395 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.630284    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0535562-96d7-4fc4-a5d5-ad4cc785080e-metrics-certs\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:26.630395 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.630322    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f26sz\" (UniqueName: \"kubernetes.io/projected/e0535562-96d7-4fc4-a5d5-ad4cc785080e-kube-api-access-f26sz\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:26.630395 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.630346    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kbb8\" (UniqueName: \"kubernetes.io/projected/d54af961-c593-4707-9cec-1069000bce50-kube-api-access-7kbb8\") pod \"kas-connection-checker-685895c7c9-nvwdl\" (UID: \"d54af961-c593-4707-9cec-1069000bce50\") " pod="kube-system/kas-connection-checker-685895c7c9-nvwdl"
Oct 30 21:37:26.631955 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.631932    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rqxfc"
Oct 30 21:37:26.650643 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.650620    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-pf9px"]
Oct 30 21:37:26.652014 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.651995    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d487d746-hf6pr"]
Oct 30 21:37:26.652674 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.652653    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-tlfjh"]
Oct 30 21:37:26.658293 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.658272    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-qqxdq"]
Oct 30 21:37:26.659922 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.659419    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q2pns"]
Oct 30 21:37:26.661310 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.660734    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-5lpq9"]
Oct 30 21:37:26.662328 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.662311    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-7tf9f"]
Oct 30 21:37:26.663352 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.663327    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-58v4l"]
Oct 30 21:37:26.681450 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.681341    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-7lnsg"]
Oct 30 21:37:26.692142 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.692115    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 30 21:37:26.692319 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.692173    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Oct 30 21:37:26.692319 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.692179    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 21:37:26.692637 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.692455    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Oct 30 21:37:26.692637 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.692501    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:37:26.694908 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.692676    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 21:37:26.694908 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.692809    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 21:37:26.694908 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.692881    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 21:37:26.694908 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.692999    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Oct 30 21:37:26.694908 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.693037    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 21:37:26.694908 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.693074    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Oct 30 21:37:26.694908 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.693148    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 21:37:26.694908 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.693189    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:37:26.694908 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.693274    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 21:37:26.694908 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.693309    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Oct 30 21:37:26.694908 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.693344    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Oct 30 21:37:26.694908 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.693383    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 21:37:26.694908 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.693514    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Oct 30 21:37:26.694908 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.693526    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 21:37:26.694908 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.693564    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 21:37:26.694908 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.693582    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 21:37:26.694908 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.693635    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:37:26.694908 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.693782    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Oct 30 21:37:26.695667 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.695261    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"]
Oct 30 21:37:26.695667 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.695318    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rqxfc"]
Oct 30 21:37:26.697591 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.697570    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:37:26.731362 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.731325    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-installation-pull-secrets\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:26.731362 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.731364    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4086d1bd-6e47-4d50-a7f9-5be23d123d6e-serving-cert\") pod \"insights-operator-7c4986d449-pf9px\" (UID: \"4086d1bd-6e47-4d50-a7f9-5be23d123d6e\") " pod="openshift-insights/insights-operator-7c4986d449-pf9px"
Oct 30 21:37:26.731549 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.731394    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e0535562-96d7-4fc4-a5d5-ad4cc785080e-stats-auth\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:26.731549 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.731466    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4qb5\" (UniqueName: \"kubernetes.io/projected/b7002d6d-834c-4bdc-907e-805edd577a40-kube-api-access-c4qb5\") pod \"console-operator-7cd56b8dbc-7tf9f\" (UID: \"b7002d6d-834c-4bdc-907e-805edd577a40\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f"
Oct 30 21:37:26.731549 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.731536    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlggr\" (UniqueName: \"kubernetes.io/projected/51e670da-3db6-412b-8753-8606d7a20fe3-kube-api-access-mlggr\") pod \"network-check-source-6f78d59f99-58v4l\" (UID: \"51e670da-3db6-412b-8753-8606d7a20fe3\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-58v4l"
Oct 30 21:37:26.731682 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.731578    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b7002d6d-834c-4bdc-907e-805edd577a40-trusted-ca\") pod \"console-operator-7cd56b8dbc-7tf9f\" (UID: \"b7002d6d-834c-4bdc-907e-805edd577a40\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f"
Oct 30 21:37:26.731682 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.731599    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/4086d1bd-6e47-4d50-a7f9-5be23d123d6e-snapshots\") pod \"insights-operator-7c4986d449-pf9px\" (UID: \"4086d1bd-6e47-4d50-a7f9-5be23d123d6e\") " pod="openshift-insights/insights-operator-7c4986d449-pf9px"
Oct 30 21:37:26.731682 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.731656    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-registry-tls\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:26.731795 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.731705    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zs5q4\" (UniqueName: \"kubernetes.io/projected/32888534-d682-4428-90d9-03c858ce5ab8-kube-api-access-zs5q4\") pod \"volume-data-source-validator-58c9f5f46d-7lnsg\" (UID: \"32888534-d682-4428-90d9-03c858ce5ab8\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-7lnsg"
Oct 30 21:37:26.731795 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.731745    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-ca-trust-extracted\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:26.731795 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.731763    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d49lh\" (UniqueName: \"kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-kube-api-access-d49lh\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:26.731795 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.731778    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5crvk\" (UniqueName: \"kubernetes.io/projected/872bccf3-b52e-4c52-9f5e-0487b4957b50-kube-api-access-5crvk\") pod \"dns-default-6q6pm\" (UID: \"872bccf3-b52e-4c52-9f5e-0487b4957b50\") " pod="openshift-dns/dns-default-6q6pm"
Oct 30 21:37:26.731944 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.731796    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0dca73b7-6526-48ac-a76e-70e441debc00-registry-certificates\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:26.731944 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.731816    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/5cf90653-8f33-4bb2-b622-1dae9144a9fe-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-q2pns\" (UID: \"5cf90653-8f33-4bb2-b622-1dae9144a9fe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q2pns"
Oct 30 21:37:26.731944 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.731844    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a099b798-61cd-4306-85ba-85c7deb25a18-nginx-conf\") pod \"networking-console-plugin-86c478bb88-5lpq9\" (UID: \"a099b798-61cd-4306-85ba-85c7deb25a18\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-5lpq9"
Oct 30 21:37:26.731944 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.731871    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-image-registry-private-configuration\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:26.732095 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.732009    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgdlc\" (UniqueName: \"kubernetes.io/projected/1dac48d8-aefb-4ff4-9213-b13def641c00-kube-api-access-kgdlc\") pod \"kas-connection-checker-685895c7c9-tlfjh\" (UID: \"1dac48d8-aefb-4ff4-9213-b13def641c00\") " pod="kube-system/kas-connection-checker-685895c7c9-tlfjh"
Oct 30 21:37:26.732095 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.732030    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4086d1bd-6e47-4d50-a7f9-5be23d123d6e-tmp\") pod \"insights-operator-7c4986d449-pf9px\" (UID: \"4086d1bd-6e47-4d50-a7f9-5be23d123d6e\") " pod="openshift-insights/insights-operator-7c4986d449-pf9px"
Oct 30 21:37:26.732193 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.732089    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn8jz\" (UniqueName: \"kubernetes.io/projected/37186e06-8d56-4c61-ba10-e0408d2e2fa4-kube-api-access-zn8jz\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-jgspb\" (UID: \"37186e06-8d56-4c61-ba10-e0408d2e2fa4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-jgspb"
Oct 30 21:37:26.732193 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.732119    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f26sz\" (UniqueName: \"kubernetes.io/projected/e0535562-96d7-4fc4-a5d5-ad4cc785080e-kube-api-access-f26sz\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:26.732193 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.732146    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b7002d6d-834c-4bdc-907e-805edd577a40-config\") pod \"console-operator-7cd56b8dbc-7tf9f\" (UID: \"b7002d6d-834c-4bdc-907e-805edd577a40\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f"
Oct 30 21:37:26.732193 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.732175    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7976h\" (UniqueName: \"kubernetes.io/projected/fb70ad86-4b45-4a53-a3cb-1f6b59640708-kube-api-access-7976h\") pod \"service-ca-operator-544d749c8c-qqxdq\" (UID: \"fb70ad86-4b45-4a53-a3cb-1f6b59640708\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-qqxdq"
Oct 30 21:37:26.732354 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.732204    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fb70ad86-4b45-4a53-a3cb-1f6b59640708-serving-cert\") pod \"service-ca-operator-544d749c8c-qqxdq\" (UID: \"fb70ad86-4b45-4a53-a3cb-1f6b59640708\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-qqxdq"
Oct 30 21:37:26.732354 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.732252    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpmml\" (UniqueName: \"kubernetes.io/projected/be4fa9bf-41e0-4b26-9457-f9815a36d5b5-kube-api-access-vpmml\") pod \"ingress-canary-rqxfc\" (UID: \"be4fa9bf-41e0-4b26-9457-f9815a36d5b5\") " pod="openshift-ingress-canary/ingress-canary-rqxfc"
Oct 30 21:37:26.732354 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.732324    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/37186e06-8d56-4c61-ba10-e0408d2e2fa4-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-jgspb\" (UID: \"37186e06-8d56-4c61-ba10-e0408d2e2fa4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-jgspb"
Oct 30 21:37:26.732354 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.732351    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8hlw\" (UniqueName: \"kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-kube-api-access-g8hlw\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:26.732479 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.732369    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmxkp\" (UniqueName: \"kubernetes.io/projected/4086d1bd-6e47-4d50-a7f9-5be23d123d6e-kube-api-access-qmxkp\") pod \"insights-operator-7c4986d449-pf9px\" (UID: \"4086d1bd-6e47-4d50-a7f9-5be23d123d6e\") " pod="openshift-insights/insights-operator-7c4986d449-pf9px"
Oct 30 21:37:26.732479 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.732424    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/be4fa9bf-41e0-4b26-9457-f9815a36d5b5-cert\") pod \"ingress-canary-rqxfc\" (UID: \"be4fa9bf-41e0-4b26-9457-f9815a36d5b5\") " pod="openshift-ingress-canary/ingress-canary-rqxfc"
Oct 30 21:37:26.732479 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.732458    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-trusted-ca\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:26.732582 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.732502    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4086d1bd-6e47-4d50-a7f9-5be23d123d6e-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-pf9px\" (UID: \"4086d1bd-6e47-4d50-a7f9-5be23d123d6e\") " pod="openshift-insights/insights-operator-7c4986d449-pf9px"
Oct 30 21:37:26.732582 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.732521    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/872bccf3-b52e-4c52-9f5e-0487b4957b50-metrics-tls\") pod \"dns-default-6q6pm\" (UID: \"872bccf3-b52e-4c52-9f5e-0487b4957b50\") " pod="openshift-dns/dns-default-6q6pm"
Oct 30 21:37:26.732582 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.732555    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/872bccf3-b52e-4c52-9f5e-0487b4957b50-tmp-dir\") pod \"dns-default-6q6pm\" (UID: \"872bccf3-b52e-4c52-9f5e-0487b4957b50\") " pod="openshift-dns/dns-default-6q6pm"
Oct 30 21:37:26.732671 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.732607    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4086d1bd-6e47-4d50-a7f9-5be23d123d6e-service-ca-bundle\") pod \"insights-operator-7c4986d449-pf9px\" (UID: \"4086d1bd-6e47-4d50-a7f9-5be23d123d6e\") " pod="openshift-insights/insights-operator-7c4986d449-pf9px"
Oct 30 21:37:26.732671 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.732656    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/37186e06-8d56-4c61-ba10-e0408d2e2fa4-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-jgspb\" (UID: \"37186e06-8d56-4c61-ba10-e0408d2e2fa4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-jgspb"
Oct 30 21:37:26.732736 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.732675    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/65827025-dc76-42d8-be9f-bdfed6b28429-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-wltlg\" (UID: \"65827025-dc76-42d8-be9f-bdfed6b28429\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wltlg"
Oct 30 21:37:26.732736 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.732692    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/872bccf3-b52e-4c52-9f5e-0487b4957b50-config-volume\") pod \"dns-default-6q6pm\" (UID: \"872bccf3-b52e-4c52-9f5e-0487b4957b50\") " pod="openshift-dns/dns-default-6q6pm"
Oct 30 21:37:26.732736 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.732711    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e0535562-96d7-4fc4-a5d5-ad4cc785080e-default-certificate\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:26.732834 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.732754    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0535562-96d7-4fc4-a5d5-ad4cc785080e-service-ca-bundle\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:26.732834 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.732776    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b7002d6d-834c-4bdc-907e-805edd577a40-serving-cert\") pod \"console-operator-7cd56b8dbc-7tf9f\" (UID: \"b7002d6d-834c-4bdc-907e-805edd577a40\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f"
Oct 30 21:37:26.732915 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:26.732897    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e0535562-96d7-4fc4-a5d5-ad4cc785080e-service-ca-bundle podName:e0535562-96d7-4fc4-a5d5-ad4cc785080e nodeName:}" failed. No retries permitted until 2026-10-30 21:37:27.232875809 +0000 UTC m=+31.559368361 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e0535562-96d7-4fc4-a5d5-ad4cc785080e-service-ca-bundle") pod "router-default-57f669cfc9-d755h" (UID: "e0535562-96d7-4fc4-a5d5-ad4cc785080e") : configmap references non-existent config key: service-ca.crt
Oct 30 21:37:26.732997 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.732973    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0dca73b7-6526-48ac-a76e-70e441debc00-image-registry-private-configuration\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:26.733116 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.733009    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-bound-sa-token\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:26.733116 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.733057    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0dca73b7-6526-48ac-a76e-70e441debc00-ca-trust-extracted\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:26.733116 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.733076    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phfjb\" (UniqueName: \"kubernetes.io/projected/5f5757a0-0a11-4637-9aa5-4e283792017d-kube-api-access-phfjb\") pod \"kas-connection-checker-685895c7c9-px9dx\" (UID: \"5f5757a0-0a11-4637-9aa5-4e283792017d\") " pod="kube-system/kas-connection-checker-685895c7c9-px9dx"
Oct 30 21:37:26.733116 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.733092    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0dca73b7-6526-48ac-a76e-70e441debc00-installation-pull-secrets\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:26.733116 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.733111    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5cf90653-8f33-4bb2-b622-1dae9144a9fe-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-q2pns\" (UID: \"5cf90653-8f33-4bb2-b622-1dae9144a9fe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q2pns"
Oct 30 21:37:26.733329 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.733139    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-certificates\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:26.733329 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.733175    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbs5d\" (UniqueName: \"kubernetes.io/projected/5cf90653-8f33-4bb2-b622-1dae9144a9fe-kube-api-access-xbs5d\") pod \"cluster-samples-operator-6ddb6c8b94-q2pns\" (UID: \"5cf90653-8f33-4bb2-b622-1dae9144a9fe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q2pns"
Oct 30 21:37:26.733329 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.733223    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a099b798-61cd-4306-85ba-85c7deb25a18-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-5lpq9\" (UID: \"a099b798-61cd-4306-85ba-85c7deb25a18\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-5lpq9"
Oct 30 21:37:26.733329 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.733292    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0535562-96d7-4fc4-a5d5-ad4cc785080e-metrics-certs\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:26.733467 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.733369    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/37186e06-8d56-4c61-ba10-e0408d2e2fa4-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-jgspb\" (UID: \"37186e06-8d56-4c61-ba10-e0408d2e2fa4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-jgspb"
Oct 30 21:37:26.733467 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.733422    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kbb8\" (UniqueName: \"kubernetes.io/projected/d54af961-c593-4707-9cec-1069000bce50-kube-api-access-7kbb8\") pod \"kas-connection-checker-685895c7c9-nvwdl\" (UID: \"d54af961-c593-4707-9cec-1069000bce50\") " pod="kube-system/kas-connection-checker-685895c7c9-nvwdl"
Oct 30 21:37:26.733535 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.733463    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0dca73b7-6526-48ac-a76e-70e441debc00-trusted-ca\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:26.733535 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.733517    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kh4k2\" (UniqueName: \"kubernetes.io/projected/65827025-dc76-42d8-be9f-bdfed6b28429-kube-api-access-kh4k2\") pod \"cluster-monitoring-operator-884576c57-wltlg\" (UID: \"65827025-dc76-42d8-be9f-bdfed6b28429\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wltlg"
Oct 30 21:37:26.733595 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.733548    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-tls\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:26.733636 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.733611    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/65827025-dc76-42d8-be9f-bdfed6b28429-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-wltlg\" (UID: \"65827025-dc76-42d8-be9f-bdfed6b28429\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wltlg"
Oct 30 21:37:26.733669 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.733638    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-bound-sa-token\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:26.733705 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.733665    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fb70ad86-4b45-4a53-a3cb-1f6b59640708-config\") pod \"service-ca-operator-544d749c8c-qqxdq\" (UID: \"fb70ad86-4b45-4a53-a3cb-1f6b59640708\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-qqxdq"
Oct 30 21:37:26.736451 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.736434    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/37186e06-8d56-4c61-ba10-e0408d2e2fa4-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-jgspb\" (UID: \"37186e06-8d56-4c61-ba10-e0408d2e2fa4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-jgspb"
Oct 30 21:37:26.738492 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.738461    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Oct 30 21:37:26.738980 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.738965    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Oct 30 21:37:26.748846 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.748815    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 30 21:37:26.748846 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.748815    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Oct 30 21:37:26.749106 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.749088    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Oct 30 21:37:26.749267 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.749217    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Oct 30 21:37:26.749333 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.749202    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Oct 30 21:37:26.749333 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.749173    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Oct 30 21:37:26.749333 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.749294    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Oct 30 21:37:26.749333 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.749328    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 21:37:26.749552 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.749366    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Oct 30 21:37:26.749552 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.749400    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Oct 30 21:37:26.749552 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.749421    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Oct 30 21:37:26.749552 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.749431    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 21:37:26.749552 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.749433    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cb4gr\""
Oct 30 21:37:26.749552 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.749439    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Oct 30 21:37:26.749552 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.749441    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-phtlh\""
Oct 30 21:37:26.749552 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.749456    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Oct 30 21:37:26.749552 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.749513    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-cds9h\""
Oct 30 21:37:26.749866 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.749750    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Oct 30 21:37:26.749866 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.749813    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-x8xlv\""
Oct 30 21:37:26.749983 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.749866    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Oct 30 21:37:26.749983 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.749901    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tsx8f\""
Oct 30 21:37:26.750120 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.750101    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Oct 30 21:37:26.750601 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.750583    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-wltlg"]
Oct 30 21:37:26.751263 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.751247    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6q6pm"]
Oct 30 21:37:26.753464 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:26.753448    2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 21:37:26.753543 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:26.753502    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0535562-96d7-4fc4-a5d5-ad4cc785080e-metrics-certs podName:e0535562-96d7-4fc4-a5d5-ad4cc785080e nodeName:}" failed. No retries permitted until 2026-10-30 21:37:27.253487287 +0000 UTC m=+31.579979832 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e0535562-96d7-4fc4-a5d5-ad4cc785080e-metrics-certs") pod "router-default-57f669cfc9-d755h" (UID: "e0535562-96d7-4fc4-a5d5-ad4cc785080e") : secret "router-metrics-certs-default" not found
Oct 30 21:37:26.754284 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.754265    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e0535562-96d7-4fc4-a5d5-ad4cc785080e-stats-auth\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:26.755250 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.755224    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e0535562-96d7-4fc4-a5d5-ad4cc785080e-default-certificate\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:26.792674 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.792642    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zn8jz\" (UniqueName: \"kubernetes.io/projected/37186e06-8d56-4c61-ba10-e0408d2e2fa4-kube-api-access-zn8jz\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-jgspb\" (UID: \"37186e06-8d56-4c61-ba10-e0408d2e2fa4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-jgspb"
Oct 30 21:37:26.792818 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.792743    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kbb8\" (UniqueName: \"kubernetes.io/projected/d54af961-c593-4707-9cec-1069000bce50-kube-api-access-7kbb8\") pod \"kas-connection-checker-685895c7c9-nvwdl\" (UID: \"d54af961-c593-4707-9cec-1069000bce50\") " pod="kube-system/kas-connection-checker-685895c7c9-nvwdl"
Oct 30 21:37:26.792818 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.792774    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f26sz\" (UniqueName: \"kubernetes.io/projected/e0535562-96d7-4fc4-a5d5-ad4cc785080e-kube-api-access-f26sz\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:26.792959 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.792930    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phfjb\" (UniqueName: \"kubernetes.io/projected/5f5757a0-0a11-4637-9aa5-4e283792017d-kube-api-access-phfjb\") pod \"kas-connection-checker-685895c7c9-px9dx\" (UID: \"5f5757a0-0a11-4637-9aa5-4e283792017d\") " pod="kube-system/kas-connection-checker-685895c7c9-px9dx"
Oct 30 21:37:26.833388 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.833345    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:26.834176 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.834155    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5cf90653-8f33-4bb2-b622-1dae9144a9fe-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-q2pns\" (UID: \"5cf90653-8f33-4bb2-b622-1dae9144a9fe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q2pns"
Oct 30 21:37:26.834273 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.834185    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-certificates\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:26.834273 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.834202    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbs5d\" (UniqueName: \"kubernetes.io/projected/5cf90653-8f33-4bb2-b622-1dae9144a9fe-kube-api-access-xbs5d\") pod \"cluster-samples-operator-6ddb6c8b94-q2pns\" (UID: \"5cf90653-8f33-4bb2-b622-1dae9144a9fe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q2pns"
Oct 30 21:37:26.834273 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.834217    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a099b798-61cd-4306-85ba-85c7deb25a18-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-5lpq9\" (UID: \"a099b798-61cd-4306-85ba-85c7deb25a18\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-5lpq9"
Oct 30 21:37:26.834411 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.834285    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0dca73b7-6526-48ac-a76e-70e441debc00-trusted-ca\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:26.834411 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.834309    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kh4k2\" (UniqueName: \"kubernetes.io/projected/65827025-dc76-42d8-be9f-bdfed6b28429-kube-api-access-kh4k2\") pod \"cluster-monitoring-operator-884576c57-wltlg\" (UID: \"65827025-dc76-42d8-be9f-bdfed6b28429\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wltlg"
Oct 30 21:37:26.834411 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:26.834313    2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 21:37:26.834411 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.834333    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-tls\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:26.834411 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:26.834370    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cf90653-8f33-4bb2-b622-1dae9144a9fe-samples-operator-tls podName:5cf90653-8f33-4bb2-b622-1dae9144a9fe nodeName:}" failed. No retries permitted until 2026-10-30 21:37:27.33435157 +0000 UTC m=+31.660844120 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5cf90653-8f33-4bb2-b622-1dae9144a9fe-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-q2pns" (UID: "5cf90653-8f33-4bb2-b622-1dae9144a9fe") : secret "samples-operator-tls" not found
Oct 30 21:37:26.834411 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.834393    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/65827025-dc76-42d8-be9f-bdfed6b28429-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-wltlg\" (UID: \"65827025-dc76-42d8-be9f-bdfed6b28429\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wltlg"
Oct 30 21:37:26.834411 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:26.834409    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 21:37:26.834739 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:26.834422    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d487d746-hf6pr: secret "image-registry-tls" not found
Oct 30 21:37:26.834739 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.834425    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-bound-sa-token\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:26.834739 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.834452    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fb70ad86-4b45-4a53-a3cb-1f6b59640708-config\") pod \"service-ca-operator-544d749c8c-qqxdq\" (UID: \"fb70ad86-4b45-4a53-a3cb-1f6b59640708\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-qqxdq"
Oct 30 21:37:26.834739 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:26.834431    2568 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 21:37:26.834739 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:26.834463    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-tls podName:30931e26-479e-4ddb-8ce3-9b2b52dc0e8e nodeName:}" failed. No retries permitted until 2026-10-30 21:37:27.334449326 +0000 UTC m=+31.660941861 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-tls") pod "image-registry-7d487d746-hf6pr" (UID: "30931e26-479e-4ddb-8ce3-9b2b52dc0e8e") : secret "image-registry-tls" not found
Oct 30 21:37:26.834739 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:26.834523    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a099b798-61cd-4306-85ba-85c7deb25a18-networking-console-plugin-cert podName:a099b798-61cd-4306-85ba-85c7deb25a18 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:27.334511314 +0000 UTC m=+31.661003848 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a099b798-61cd-4306-85ba-85c7deb25a18-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-5lpq9" (UID: "a099b798-61cd-4306-85ba-85c7deb25a18") : secret "networking-console-plugin-cert" not found
Oct 30 21:37:26.834739 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.834562    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-installation-pull-secrets\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:26.834739 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.834586    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4086d1bd-6e47-4d50-a7f9-5be23d123d6e-serving-cert\") pod \"insights-operator-7c4986d449-pf9px\" (UID: \"4086d1bd-6e47-4d50-a7f9-5be23d123d6e\") " pod="openshift-insights/insights-operator-7c4986d449-pf9px"
Oct 30 21:37:26.834739 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.834618    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4qb5\" (UniqueName: \"kubernetes.io/projected/b7002d6d-834c-4bdc-907e-805edd577a40-kube-api-access-c4qb5\") pod \"console-operator-7cd56b8dbc-7tf9f\" (UID: \"b7002d6d-834c-4bdc-907e-805edd577a40\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f"
Oct 30 21:37:26.834739 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.834648    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlggr\" (UniqueName: \"kubernetes.io/projected/51e670da-3db6-412b-8753-8606d7a20fe3-kube-api-access-mlggr\") pod \"network-check-source-6f78d59f99-58v4l\" (UID: \"51e670da-3db6-412b-8753-8606d7a20fe3\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-58v4l"
Oct 30 21:37:26.834739 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.834677    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b7002d6d-834c-4bdc-907e-805edd577a40-trusted-ca\") pod \"console-operator-7cd56b8dbc-7tf9f\" (UID: \"b7002d6d-834c-4bdc-907e-805edd577a40\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f"
Oct 30 21:37:26.834739 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.834702    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/4086d1bd-6e47-4d50-a7f9-5be23d123d6e-snapshots\") pod \"insights-operator-7c4986d449-pf9px\" (UID: \"4086d1bd-6e47-4d50-a7f9-5be23d123d6e\") " pod="openshift-insights/insights-operator-7c4986d449-pf9px"
Oct 30 21:37:26.834739 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.834735    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-registry-tls\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:26.835378 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.834776    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zs5q4\" (UniqueName: \"kubernetes.io/projected/32888534-d682-4428-90d9-03c858ce5ab8-kube-api-access-zs5q4\") pod \"volume-data-source-validator-58c9f5f46d-7lnsg\" (UID: \"32888534-d682-4428-90d9-03c858ce5ab8\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-7lnsg"
Oct 30 21:37:26.835378 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.834800    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-ca-trust-extracted\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:26.835378 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.834825    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d49lh\" (UniqueName: \"kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-kube-api-access-d49lh\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:26.835378 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.834849    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5crvk\" (UniqueName: \"kubernetes.io/projected/872bccf3-b52e-4c52-9f5e-0487b4957b50-kube-api-access-5crvk\") pod \"dns-default-6q6pm\" (UID: \"872bccf3-b52e-4c52-9f5e-0487b4957b50\") " pod="openshift-dns/dns-default-6q6pm"
Oct 30 21:37:26.835378 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.834877    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0dca73b7-6526-48ac-a76e-70e441debc00-registry-certificates\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:26.835378 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.834903    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/5cf90653-8f33-4bb2-b622-1dae9144a9fe-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-q2pns\" (UID: \"5cf90653-8f33-4bb2-b622-1dae9144a9fe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q2pns"
Oct 30 21:37:26.835378 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.834926    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a099b798-61cd-4306-85ba-85c7deb25a18-nginx-conf\") pod \"networking-console-plugin-86c478bb88-5lpq9\" (UID: \"a099b798-61cd-4306-85ba-85c7deb25a18\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-5lpq9"
Oct 30 21:37:26.835378 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.834955    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-image-registry-private-configuration\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:26.835378 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.835014    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgdlc\" (UniqueName: \"kubernetes.io/projected/1dac48d8-aefb-4ff4-9213-b13def641c00-kube-api-access-kgdlc\") pod \"kas-connection-checker-685895c7c9-tlfjh\" (UID: \"1dac48d8-aefb-4ff4-9213-b13def641c00\") " pod="kube-system/kas-connection-checker-685895c7c9-tlfjh"
Oct 30 21:37:26.835378 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.835038    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4086d1bd-6e47-4d50-a7f9-5be23d123d6e-tmp\") pod \"insights-operator-7c4986d449-pf9px\" (UID: \"4086d1bd-6e47-4d50-a7f9-5be23d123d6e\") " pod="openshift-insights/insights-operator-7c4986d449-pf9px"
Oct 30 21:37:26.835378 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.835083    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b7002d6d-834c-4bdc-907e-805edd577a40-config\") pod \"console-operator-7cd56b8dbc-7tf9f\" (UID: \"b7002d6d-834c-4bdc-907e-805edd577a40\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f"
Oct 30 21:37:26.835378 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.835084    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fb70ad86-4b45-4a53-a3cb-1f6b59640708-config\") pod \"service-ca-operator-544d749c8c-qqxdq\" (UID: \"fb70ad86-4b45-4a53-a3cb-1f6b59640708\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-qqxdq"
Oct 30 21:37:26.835378 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.835109    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7976h\" (UniqueName: \"kubernetes.io/projected/fb70ad86-4b45-4a53-a3cb-1f6b59640708-kube-api-access-7976h\") pod \"service-ca-operator-544d749c8c-qqxdq\" (UID: \"fb70ad86-4b45-4a53-a3cb-1f6b59640708\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-qqxdq"
Oct 30 21:37:26.835378 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.835145    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fb70ad86-4b45-4a53-a3cb-1f6b59640708-serving-cert\") pod \"service-ca-operator-544d749c8c-qqxdq\" (UID: \"fb70ad86-4b45-4a53-a3cb-1f6b59640708\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-qqxdq"
Oct 30 21:37:26.835378 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.835149    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-certificates\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:26.835378 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.835174    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpmml\" (UniqueName: \"kubernetes.io/projected/be4fa9bf-41e0-4b26-9457-f9815a36d5b5-kube-api-access-vpmml\") pod \"ingress-canary-rqxfc\" (UID: \"be4fa9bf-41e0-4b26-9457-f9815a36d5b5\") " pod="openshift-ingress-canary/ingress-canary-rqxfc"
Oct 30 21:37:26.836109 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.835204    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8hlw\" (UniqueName: \"kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-kube-api-access-g8hlw\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:26.836109 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.835228    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmxkp\" (UniqueName: \"kubernetes.io/projected/4086d1bd-6e47-4d50-a7f9-5be23d123d6e-kube-api-access-qmxkp\") pod \"insights-operator-7c4986d449-pf9px\" (UID: \"4086d1bd-6e47-4d50-a7f9-5be23d123d6e\") " pod="openshift-insights/insights-operator-7c4986d449-pf9px"
Oct 30 21:37:26.836109 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.835275    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/be4fa9bf-41e0-4b26-9457-f9815a36d5b5-cert\") pod \"ingress-canary-rqxfc\" (UID: \"be4fa9bf-41e0-4b26-9457-f9815a36d5b5\") " pod="openshift-ingress-canary/ingress-canary-rqxfc"
Oct 30 21:37:26.836109 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.835293    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-trusted-ca\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:26.836109 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.835293    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/65827025-dc76-42d8-be9f-bdfed6b28429-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-wltlg\" (UID: \"65827025-dc76-42d8-be9f-bdfed6b28429\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wltlg"
Oct 30 21:37:26.836109 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.835311    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4086d1bd-6e47-4d50-a7f9-5be23d123d6e-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-pf9px\" (UID: \"4086d1bd-6e47-4d50-a7f9-5be23d123d6e\") " pod="openshift-insights/insights-operator-7c4986d449-pf9px"
Oct 30 21:37:26.836109 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.835328    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/872bccf3-b52e-4c52-9f5e-0487b4957b50-metrics-tls\") pod \"dns-default-6q6pm\" (UID: \"872bccf3-b52e-4c52-9f5e-0487b4957b50\") " pod="openshift-dns/dns-default-6q6pm"
Oct 30 21:37:26.836109 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.835343    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/872bccf3-b52e-4c52-9f5e-0487b4957b50-tmp-dir\") pod \"dns-default-6q6pm\" (UID: \"872bccf3-b52e-4c52-9f5e-0487b4957b50\") " pod="openshift-dns/dns-default-6q6pm"
Oct 30 21:37:26.836109 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.835365    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4086d1bd-6e47-4d50-a7f9-5be23d123d6e-service-ca-bundle\") pod \"insights-operator-7c4986d449-pf9px\" (UID: \"4086d1bd-6e47-4d50-a7f9-5be23d123d6e\") " pod="openshift-insights/insights-operator-7c4986d449-pf9px"
Oct 30 21:37:26.836109 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.835386    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/65827025-dc76-42d8-be9f-bdfed6b28429-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-wltlg\" (UID: \"65827025-dc76-42d8-be9f-bdfed6b28429\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wltlg"
Oct 30 21:37:26.836109 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:26.835400    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 21:37:26.836109 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:26.835414    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55fbc4d8d7-w4mrq: secret "image-registry-tls" not found
Oct 30 21:37:26.836109 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:26.835457    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-registry-tls podName:0dca73b7-6526-48ac-a76e-70e441debc00 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:27.335443796 +0000 UTC m=+31.661936329 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-registry-tls") pod "image-registry-55fbc4d8d7-w4mrq" (UID: "0dca73b7-6526-48ac-a76e-70e441debc00") : secret "image-registry-tls" not found
Oct 30 21:37:26.836109 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.835460    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0dca73b7-6526-48ac-a76e-70e441debc00-trusted-ca\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:26.836109 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.835402    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/872bccf3-b52e-4c52-9f5e-0487b4957b50-config-volume\") pod \"dns-default-6q6pm\" (UID: \"872bccf3-b52e-4c52-9f5e-0487b4957b50\") " pod="openshift-dns/dns-default-6q6pm"
Oct 30 21:37:26.836109 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.835656    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0dca73b7-6526-48ac-a76e-70e441debc00-registry-certificates\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:26.836109 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.836109    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/4086d1bd-6e47-4d50-a7f9-5be23d123d6e-snapshots\") pod \"insights-operator-7c4986d449-pf9px\" (UID: \"4086d1bd-6e47-4d50-a7f9-5be23d123d6e\") " pod="openshift-insights/insights-operator-7c4986d449-pf9px"
Oct 30 21:37:26.837043 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:26.836200    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:37:26.837043 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:26.836262    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/872bccf3-b52e-4c52-9f5e-0487b4957b50-metrics-tls podName:872bccf3-b52e-4c52-9f5e-0487b4957b50 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:27.336227968 +0000 UTC m=+31.662720506 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/872bccf3-b52e-4c52-9f5e-0487b4957b50-metrics-tls") pod "dns-default-6q6pm" (UID: "872bccf3-b52e-4c52-9f5e-0487b4957b50") : secret "dns-default-metrics-tls" not found
Oct 30 21:37:26.837043 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.836389    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-ca-trust-extracted\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:26.837043 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.836443    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4086d1bd-6e47-4d50-a7f9-5be23d123d6e-tmp\") pod \"insights-operator-7c4986d449-pf9px\" (UID: \"4086d1bd-6e47-4d50-a7f9-5be23d123d6e\") " pod="openshift-insights/insights-operator-7c4986d449-pf9px"
Oct 30 21:37:26.837043 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.836461    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b7002d6d-834c-4bdc-907e-805edd577a40-trusted-ca\") pod \"console-operator-7cd56b8dbc-7tf9f\" (UID: \"b7002d6d-834c-4bdc-907e-805edd577a40\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f"
Oct 30 21:37:26.837043 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.836510    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/872bccf3-b52e-4c52-9f5e-0487b4957b50-tmp-dir\") pod \"dns-default-6q6pm\" (UID: \"872bccf3-b52e-4c52-9f5e-0487b4957b50\") " pod="openshift-dns/dns-default-6q6pm"
Oct 30 21:37:26.837043 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.836755    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/5cf90653-8f33-4bb2-b622-1dae9144a9fe-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-q2pns\" (UID: \"5cf90653-8f33-4bb2-b622-1dae9144a9fe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q2pns"
Oct 30 21:37:26.837043 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:26.836842    2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 21:37:26.837043 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:26.836884    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65827025-dc76-42d8-be9f-bdfed6b28429-cluster-monitoring-operator-tls podName:65827025-dc76-42d8-be9f-bdfed6b28429 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:27.33687123 +0000 UTC m=+31.663363765 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/65827025-dc76-42d8-be9f-bdfed6b28429-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-wltlg" (UID: "65827025-dc76-42d8-be9f-bdfed6b28429") : secret "cluster-monitoring-operator-tls" not found
Oct 30 21:37:26.837043 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.836921    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4086d1bd-6e47-4d50-a7f9-5be23d123d6e-service-ca-bundle\") pod \"insights-operator-7c4986d449-pf9px\" (UID: \"4086d1bd-6e47-4d50-a7f9-5be23d123d6e\") " pod="openshift-insights/insights-operator-7c4986d449-pf9px"
Oct 30 21:37:26.837043 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.836954    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b7002d6d-834c-4bdc-907e-805edd577a40-serving-cert\") pod \"console-operator-7cd56b8dbc-7tf9f\" (UID: \"b7002d6d-834c-4bdc-907e-805edd577a40\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f"
Oct 30 21:37:26.837043 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.836998    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0dca73b7-6526-48ac-a76e-70e441debc00-image-registry-private-configuration\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:26.837043 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.837028    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-bound-sa-token\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:26.837699 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.837052    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b7002d6d-834c-4bdc-907e-805edd577a40-config\") pod \"console-operator-7cd56b8dbc-7tf9f\" (UID: \"b7002d6d-834c-4bdc-907e-805edd577a40\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f"
Oct 30 21:37:26.837699 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.837065    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0dca73b7-6526-48ac-a76e-70e441debc00-ca-trust-extracted\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:26.837699 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.837105    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0dca73b7-6526-48ac-a76e-70e441debc00-installation-pull-secrets\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:26.837699 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.837593    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0dca73b7-6526-48ac-a76e-70e441debc00-ca-trust-extracted\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:26.837699 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.837663    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4086d1bd-6e47-4d50-a7f9-5be23d123d6e-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-pf9px\" (UID: \"4086d1bd-6e47-4d50-a7f9-5be23d123d6e\") " pod="openshift-insights/insights-operator-7c4986d449-pf9px"
Oct 30 21:37:26.837929 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.837766    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-trusted-ca\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:26.837978 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.837940    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/872bccf3-b52e-4c52-9f5e-0487b4957b50-config-volume\") pod \"dns-default-6q6pm\" (UID: \"872bccf3-b52e-4c52-9f5e-0487b4957b50\") " pod="openshift-dns/dns-default-6q6pm"
Oct 30 21:37:26.838078 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.838057    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a099b798-61cd-4306-85ba-85c7deb25a18-nginx-conf\") pod \"networking-console-plugin-86c478bb88-5lpq9\" (UID: \"a099b798-61cd-4306-85ba-85c7deb25a18\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-5lpq9"
Oct 30 21:37:26.839045 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.839005    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4086d1bd-6e47-4d50-a7f9-5be23d123d6e-serving-cert\") pod \"insights-operator-7c4986d449-pf9px\" (UID: \"4086d1bd-6e47-4d50-a7f9-5be23d123d6e\") " pod="openshift-insights/insights-operator-7c4986d449-pf9px"
Oct 30 21:37:26.839354 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.839333    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-installation-pull-secrets\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:26.839459 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.839439    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fb70ad86-4b45-4a53-a3cb-1f6b59640708-serving-cert\") pod \"service-ca-operator-544d749c8c-qqxdq\" (UID: \"fb70ad86-4b45-4a53-a3cb-1f6b59640708\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-qqxdq"
Oct 30 21:37:26.839913 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.839895    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0dca73b7-6526-48ac-a76e-70e441debc00-image-registry-private-configuration\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:26.840145 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.840128    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0dca73b7-6526-48ac-a76e-70e441debc00-installation-pull-secrets\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:26.840204 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.840192    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-image-registry-private-configuration\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:26.840313 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.840297    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/be4fa9bf-41e0-4b26-9457-f9815a36d5b5-cert\") pod \"ingress-canary-rqxfc\" (UID: \"be4fa9bf-41e0-4b26-9457-f9815a36d5b5\") " pod="openshift-ingress-canary/ingress-canary-rqxfc"
Oct 30 21:37:26.840768 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.840750    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b7002d6d-834c-4bdc-907e-805edd577a40-serving-cert\") pod \"console-operator-7cd56b8dbc-7tf9f\" (UID: \"b7002d6d-834c-4bdc-907e-805edd577a40\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f"
Oct 30 21:37:26.853806 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.853776    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-px9dx"
Oct 30 21:37:26.858953 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.858924    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-jgspb"
Oct 30 21:37:26.864515 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.864498    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-nvwdl"
Oct 30 21:37:26.895077 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.894994    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbs5d\" (UniqueName: \"kubernetes.io/projected/5cf90653-8f33-4bb2-b622-1dae9144a9fe-kube-api-access-xbs5d\") pod \"cluster-samples-operator-6ddb6c8b94-q2pns\" (UID: \"5cf90653-8f33-4bb2-b622-1dae9144a9fe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q2pns"
Oct 30 21:37:26.895983 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.895955    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4qb5\" (UniqueName: \"kubernetes.io/projected/b7002d6d-834c-4bdc-907e-805edd577a40-kube-api-access-c4qb5\") pod \"console-operator-7cd56b8dbc-7tf9f\" (UID: \"b7002d6d-834c-4bdc-907e-805edd577a40\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f"
Oct 30 21:37:26.896744 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.896722    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-bound-sa-token\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:26.897585 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.897475    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlggr\" (UniqueName: \"kubernetes.io/projected/51e670da-3db6-412b-8753-8606d7a20fe3-kube-api-access-mlggr\") pod \"network-check-source-6f78d59f99-58v4l\" (UID: \"51e670da-3db6-412b-8753-8606d7a20fe3\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-58v4l"
Oct 30 21:37:26.901724 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.901177    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmxkp\" (UniqueName: \"kubernetes.io/projected/4086d1bd-6e47-4d50-a7f9-5be23d123d6e-kube-api-access-qmxkp\") pod \"insights-operator-7c4986d449-pf9px\" (UID: \"4086d1bd-6e47-4d50-a7f9-5be23d123d6e\") " pod="openshift-insights/insights-operator-7c4986d449-pf9px"
Oct 30 21:37:26.901724 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.901391    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kh4k2\" (UniqueName: \"kubernetes.io/projected/65827025-dc76-42d8-be9f-bdfed6b28429-kube-api-access-kh4k2\") pod \"cluster-monitoring-operator-884576c57-wltlg\" (UID: \"65827025-dc76-42d8-be9f-bdfed6b28429\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wltlg"
Oct 30 21:37:26.902877 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.902847    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zs5q4\" (UniqueName: \"kubernetes.io/projected/32888534-d682-4428-90d9-03c858ce5ab8-kube-api-access-zs5q4\") pod \"volume-data-source-validator-58c9f5f46d-7lnsg\" (UID: \"32888534-d682-4428-90d9-03c858ce5ab8\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-7lnsg"
Oct 30 21:37:26.904445 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.904315    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpmml\" (UniqueName: \"kubernetes.io/projected/be4fa9bf-41e0-4b26-9457-f9815a36d5b5-kube-api-access-vpmml\") pod \"ingress-canary-rqxfc\" (UID: \"be4fa9bf-41e0-4b26-9457-f9815a36d5b5\") " pod="openshift-ingress-canary/ingress-canary-rqxfc"
Oct 30 21:37:26.904445 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.904350    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8hlw\" (UniqueName: \"kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-kube-api-access-g8hlw\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:26.905014 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.904866    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d49lh\" (UniqueName: \"kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-kube-api-access-d49lh\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:26.905014 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.904962    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5crvk\" (UniqueName: \"kubernetes.io/projected/872bccf3-b52e-4c52-9f5e-0487b4957b50-kube-api-access-5crvk\") pod \"dns-default-6q6pm\" (UID: \"872bccf3-b52e-4c52-9f5e-0487b4957b50\") " pod="openshift-dns/dns-default-6q6pm"
Oct 30 21:37:26.905532 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.905493    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7976h\" (UniqueName: \"kubernetes.io/projected/fb70ad86-4b45-4a53-a3cb-1f6b59640708-kube-api-access-7976h\") pod \"service-ca-operator-544d749c8c-qqxdq\" (UID: \"fb70ad86-4b45-4a53-a3cb-1f6b59640708\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-qqxdq"
Oct 30 21:37:26.906641 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.906618    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgdlc\" (UniqueName: \"kubernetes.io/projected/1dac48d8-aefb-4ff4-9213-b13def641c00-kube-api-access-kgdlc\") pod \"kas-connection-checker-685895c7c9-tlfjh\" (UID: \"1dac48d8-aefb-4ff4-9213-b13def641c00\") " pod="kube-system/kas-connection-checker-685895c7c9-tlfjh"
Oct 30 21:37:26.906692 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.906198    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-bound-sa-token\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:26.913162 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.912806    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-pf9px"
Oct 30 21:37:26.915961 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.915942    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-qqxdq"
Oct 30 21:37:26.963284 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.958059    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-7lnsg"
Oct 30 21:37:26.966972 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.966872    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f"
Oct 30 21:37:26.977442 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.977245    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-58v4l"
Oct 30 21:37:26.989577 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:26.989041    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rqxfc"
Oct 30 21:37:27.061656 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:27.061622    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:27.201302 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:27.201196    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-tlfjh"
Oct 30 21:37:27.241365 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:27.241341    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0535562-96d7-4fc4-a5d5-ad4cc785080e-service-ca-bundle\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:27.241680 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:27.241506    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e0535562-96d7-4fc4-a5d5-ad4cc785080e-service-ca-bundle podName:e0535562-96d7-4fc4-a5d5-ad4cc785080e nodeName:}" failed. No retries permitted until 2026-10-30 21:37:28.241489242 +0000 UTC m=+32.567981793 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e0535562-96d7-4fc4-a5d5-ad4cc785080e-service-ca-bundle") pod "router-default-57f669cfc9-d755h" (UID: "e0535562-96d7-4fc4-a5d5-ad4cc785080e") : configmap references non-existent config key: service-ca.crt
Oct 30 21:37:27.260788 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:27.260740    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-jgspb"]
Oct 30 21:37:27.260950 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:27.260820    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-nvwdl"]
Oct 30 21:37:27.262501 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:27.262475    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-px9dx"]
Oct 30 21:37:27.267412 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:37:27.266193    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd54af961_c593_4707_9cec_1069000bce50.slice/crio-1d9a1d9c4bcaff3bf006aa1fd82d5f0ed5473e5536c00f96aa2ab48006a271eb WatchSource:0}: Error finding container 1d9a1d9c4bcaff3bf006aa1fd82d5f0ed5473e5536c00f96aa2ab48006a271eb: Status 404 returned error can't find the container with id 1d9a1d9c4bcaff3bf006aa1fd82d5f0ed5473e5536c00f96aa2ab48006a271eb
Oct 30 21:37:27.331377 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:27.331291    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-qqxdq"]
Oct 30 21:37:27.332032 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:27.332010    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-pf9px"]
Oct 30 21:37:27.332557 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:37:27.332524    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb70ad86_4b45_4a53_a3cb_1f6b59640708.slice/crio-e67ac7ed16cc9130b49c76aece77d8e8eec0b37d7eccde37c096a9c04de6fbe1 WatchSource:0}: Error finding container e67ac7ed16cc9130b49c76aece77d8e8eec0b37d7eccde37c096a9c04de6fbe1: Status 404 returned error can't find the container with id e67ac7ed16cc9130b49c76aece77d8e8eec0b37d7eccde37c096a9c04de6fbe1
Oct 30 21:37:27.334649 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:37:27.334610    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4086d1bd_6e47_4d50_a7f9_5be23d123d6e.slice/crio-3c5ad66e5790dfd6cd348d4f2ae9c2c4418f37047a4a9e8a5cf4de6772f91fc6 WatchSource:0}: Error finding container 3c5ad66e5790dfd6cd348d4f2ae9c2c4418f37047a4a9e8a5cf4de6772f91fc6: Status 404 returned error can't find the container with id 3c5ad66e5790dfd6cd348d4f2ae9c2c4418f37047a4a9e8a5cf4de6772f91fc6
Oct 30 21:37:27.342173 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:27.342147    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/65827025-dc76-42d8-be9f-bdfed6b28429-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-wltlg\" (UID: \"65827025-dc76-42d8-be9f-bdfed6b28429\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wltlg"
Oct 30 21:37:27.342292 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:27.342217    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5cf90653-8f33-4bb2-b622-1dae9144a9fe-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-q2pns\" (UID: \"5cf90653-8f33-4bb2-b622-1dae9144a9fe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q2pns"
Oct 30 21:37:27.342292 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:27.342281    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a099b798-61cd-4306-85ba-85c7deb25a18-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-5lpq9\" (UID: \"a099b798-61cd-4306-85ba-85c7deb25a18\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-5lpq9"
Oct 30 21:37:27.342405 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:27.342325    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0535562-96d7-4fc4-a5d5-ad4cc785080e-metrics-certs\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:27.342405 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:27.342364    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-tls\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:27.342507 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:27.342411    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-registry-tls\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:27.342507 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:27.342498    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/872bccf3-b52e-4c52-9f5e-0487b4957b50-metrics-tls\") pod \"dns-default-6q6pm\" (UID: \"872bccf3-b52e-4c52-9f5e-0487b4957b50\") " pod="openshift-dns/dns-default-6q6pm"
Oct 30 21:37:27.342665 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:27.342646    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:37:27.342717 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:27.342704    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/872bccf3-b52e-4c52-9f5e-0487b4957b50-metrics-tls podName:872bccf3-b52e-4c52-9f5e-0487b4957b50 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:28.342686314 +0000 UTC m=+32.669178848 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/872bccf3-b52e-4c52-9f5e-0487b4957b50-metrics-tls") pod "dns-default-6q6pm" (UID: "872bccf3-b52e-4c52-9f5e-0487b4957b50") : secret "dns-default-metrics-tls" not found
Oct 30 21:37:27.342792 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:27.342769    2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 21:37:27.342872 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:27.342811    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65827025-dc76-42d8-be9f-bdfed6b28429-cluster-monitoring-operator-tls podName:65827025-dc76-42d8-be9f-bdfed6b28429 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:28.342803249 +0000 UTC m=+32.669295780 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/65827025-dc76-42d8-be9f-bdfed6b28429-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-wltlg" (UID: "65827025-dc76-42d8-be9f-bdfed6b28429") : secret "cluster-monitoring-operator-tls" not found
Oct 30 21:37:27.342872 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:27.342857    2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 21:37:27.342981 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:27.342888    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cf90653-8f33-4bb2-b622-1dae9144a9fe-samples-operator-tls podName:5cf90653-8f33-4bb2-b622-1dae9144a9fe nodeName:}" failed. No retries permitted until 2026-10-30 21:37:28.342878427 +0000 UTC m=+32.669370960 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5cf90653-8f33-4bb2-b622-1dae9144a9fe-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-q2pns" (UID: "5cf90653-8f33-4bb2-b622-1dae9144a9fe") : secret "samples-operator-tls" not found
Oct 30 21:37:27.342981 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:27.342937    2568 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 21:37:27.342981 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:27.342966    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a099b798-61cd-4306-85ba-85c7deb25a18-networking-console-plugin-cert podName:a099b798-61cd-4306-85ba-85c7deb25a18 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:28.342956318 +0000 UTC m=+32.669448852 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a099b798-61cd-4306-85ba-85c7deb25a18-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-5lpq9" (UID: "a099b798-61cd-4306-85ba-85c7deb25a18") : secret "networking-console-plugin-cert" not found
Oct 30 21:37:27.343126 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:27.343020    2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 21:37:27.343126 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:27.343052    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0535562-96d7-4fc4-a5d5-ad4cc785080e-metrics-certs podName:e0535562-96d7-4fc4-a5d5-ad4cc785080e nodeName:}" failed. No retries permitted until 2026-10-30 21:37:28.343043615 +0000 UTC m=+32.669536163 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e0535562-96d7-4fc4-a5d5-ad4cc785080e-metrics-certs") pod "router-default-57f669cfc9-d755h" (UID: "e0535562-96d7-4fc4-a5d5-ad4cc785080e") : secret "router-metrics-certs-default" not found
Oct 30 21:37:27.343126 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:27.343115    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 21:37:27.343126 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:27.343126    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d487d746-hf6pr: secret "image-registry-tls" not found
Oct 30 21:37:27.343328 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:27.343154    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-tls podName:30931e26-479e-4ddb-8ce3-9b2b52dc0e8e nodeName:}" failed. No retries permitted until 2026-10-30 21:37:28.343144923 +0000 UTC m=+32.669637458 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-tls") pod "image-registry-7d487d746-hf6pr" (UID: "30931e26-479e-4ddb-8ce3-9b2b52dc0e8e") : secret "image-registry-tls" not found
Oct 30 21:37:27.343328 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:27.343206    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 21:37:27.343328 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:27.343216    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55fbc4d8d7-w4mrq: secret "image-registry-tls" not found
Oct 30 21:37:27.343328 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:27.343256    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-registry-tls podName:0dca73b7-6526-48ac-a76e-70e441debc00 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:28.343232926 +0000 UTC m=+32.669725457 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-registry-tls") pod "image-registry-55fbc4d8d7-w4mrq" (UID: "0dca73b7-6526-48ac-a76e-70e441debc00") : secret "image-registry-tls" not found
Oct 30 21:37:27.373919 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:27.373895    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-7tf9f"]
Oct 30 21:37:27.376952 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:37:27.376925    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7002d6d_834c_4bdc_907e_805edd577a40.slice/crio-05c69dcfb41fe3181298c0037998566ca8c95db99104a8739c64cc106cff8214 WatchSource:0}: Error finding container 05c69dcfb41fe3181298c0037998566ca8c95db99104a8739c64cc106cff8214: Status 404 returned error can't find the container with id 05c69dcfb41fe3181298c0037998566ca8c95db99104a8739c64cc106cff8214
Oct 30 21:37:27.383519 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:27.383425    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-7lnsg"]
Oct 30 21:37:27.386119 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:37:27.386099    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32888534_d682_4428_90d9_03c858ce5ab8.slice/crio-168e06a666a628bbe4b646281ae2c3b727dbb614786968762d945ffc95c1d908 WatchSource:0}: Error finding container 168e06a666a628bbe4b646281ae2c3b727dbb614786968762d945ffc95c1d908: Status 404 returned error can't find the container with id 168e06a666a628bbe4b646281ae2c3b727dbb614786968762d945ffc95c1d908
Oct 30 21:37:27.391971 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:27.391947    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rqxfc"]
Oct 30 21:37:27.395009 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:37:27.394986    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe4fa9bf_41e0_4b26_9457_f9815a36d5b5.slice/crio-92c0b7873904ff975ba9638df3112650d5becec57db5160005d933c8de8860cd WatchSource:0}: Error finding container 92c0b7873904ff975ba9638df3112650d5becec57db5160005d933c8de8860cd: Status 404 returned error can't find the container with id 92c0b7873904ff975ba9638df3112650d5becec57db5160005d933c8de8860cd
Oct 30 21:37:27.405114 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:27.405088    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f" event={"ID":"b7002d6d-834c-4bdc-907e-805edd577a40","Type":"ContainerStarted","Data":"05c69dcfb41fe3181298c0037998566ca8c95db99104a8739c64cc106cff8214"}
Oct 30 21:37:27.406272 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:27.406229    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-px9dx" event={"ID":"5f5757a0-0a11-4637-9aa5-4e283792017d","Type":"ContainerStarted","Data":"722f567ea5aca5dab8d18f3865b9a6a56f2a0a9123efa1d3bd7a8efee9bc2d91"}
Oct 30 21:37:27.407280 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:27.407259    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-pf9px" event={"ID":"4086d1bd-6e47-4d50-a7f9-5be23d123d6e","Type":"ContainerStarted","Data":"3c5ad66e5790dfd6cd348d4f2ae9c2c4418f37047a4a9e8a5cf4de6772f91fc6"}
Oct 30 21:37:27.408276 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:27.408233    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-nvwdl" event={"ID":"d54af961-c593-4707-9cec-1069000bce50","Type":"ContainerStarted","Data":"1d9a1d9c4bcaff3bf006aa1fd82d5f0ed5473e5536c00f96aa2ab48006a271eb"}
Oct 30 21:37:27.409280 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:27.409259    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rqxfc" event={"ID":"be4fa9bf-41e0-4b26-9457-f9815a36d5b5","Type":"ContainerStarted","Data":"92c0b7873904ff975ba9638df3112650d5becec57db5160005d933c8de8860cd"}
Oct 30 21:37:27.410403 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:27.410379    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-7lnsg" event={"ID":"32888534-d682-4428-90d9-03c858ce5ab8","Type":"ContainerStarted","Data":"168e06a666a628bbe4b646281ae2c3b727dbb614786968762d945ffc95c1d908"}
Oct 30 21:37:27.412987 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:27.411726    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-qqxdq" event={"ID":"fb70ad86-4b45-4a53-a3cb-1f6b59640708","Type":"ContainerStarted","Data":"e67ac7ed16cc9130b49c76aece77d8e8eec0b37d7eccde37c096a9c04de6fbe1"}
Oct 30 21:37:27.412987 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:27.412718    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-58v4l"]
Oct 30 21:37:27.413128 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:27.413110    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-jgspb" event={"ID":"37186e06-8d56-4c61-ba10-e0408d2e2fa4","Type":"ContainerStarted","Data":"4ebb73f893610de7c7f7c99d369364e94cee977c267a33f918b1ffed7e1d6963"}
Oct 30 21:37:27.417054 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:37:27.417032    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51e670da_3db6_412b_8753_8606d7a20fe3.slice/crio-8141d1060aedcae9e8276e2435ae0d51406506284af41b835416efde5b1f99fa WatchSource:0}: Error finding container 8141d1060aedcae9e8276e2435ae0d51406506284af41b835416efde5b1f99fa: Status 404 returned error can't find the container with id 8141d1060aedcae9e8276e2435ae0d51406506284af41b835416efde5b1f99fa
Oct 30 21:37:27.592910 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:27.592834    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-tlfjh"]
Oct 30 21:37:27.595658 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:37:27.595628    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dac48d8_aefb_4ff4_9213_b13def641c00.slice/crio-047c43343c8d3a6ca7dbb44be20d27b13bc9d3a62d2c27732b32c88271a79021 WatchSource:0}: Error finding container 047c43343c8d3a6ca7dbb44be20d27b13bc9d3a62d2c27732b32c88271a79021: Status 404 returned error can't find the container with id 047c43343c8d3a6ca7dbb44be20d27b13bc9d3a62d2c27732b32c88271a79021
Oct 30 21:37:28.220113 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:28.220090    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9kxl"
Oct 30 21:37:28.220277 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:28.220176    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wpc6f"
Oct 30 21:37:28.220340 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:28.220288    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:37:28.249316 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:28.249290    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0535562-96d7-4fc4-a5d5-ad4cc785080e-service-ca-bundle\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:28.249624 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:28.249460    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e0535562-96d7-4fc4-a5d5-ad4cc785080e-service-ca-bundle podName:e0535562-96d7-4fc4-a5d5-ad4cc785080e nodeName:}" failed. No retries permitted until 2026-10-30 21:37:30.249441726 +0000 UTC m=+34.575934278 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e0535562-96d7-4fc4-a5d5-ad4cc785080e-service-ca-bundle") pod "router-default-57f669cfc9-d755h" (UID: "e0535562-96d7-4fc4-a5d5-ad4cc785080e") : configmap references non-existent config key: service-ca.crt
Oct 30 21:37:28.272554 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:28.272522    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 21:37:28.272554 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:28.272548    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-x7gkt\""
Oct 30 21:37:28.272716 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:28.272530    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-ktl9g\""
Oct 30 21:37:28.272716 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:28.272523    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-v9xch\""
Oct 30 21:37:28.272716 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:28.272699    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 21:37:28.349869 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:28.349832    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-registry-tls\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:28.350008 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:28.349903    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/872bccf3-b52e-4c52-9f5e-0487b4957b50-metrics-tls\") pod \"dns-default-6q6pm\" (UID: \"872bccf3-b52e-4c52-9f5e-0487b4957b50\") " pod="openshift-dns/dns-default-6q6pm"
Oct 30 21:37:28.350008 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:28.349940    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/65827025-dc76-42d8-be9f-bdfed6b28429-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-wltlg\" (UID: \"65827025-dc76-42d8-be9f-bdfed6b28429\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wltlg"
Oct 30 21:37:28.350008 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:28.349988    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5cf90653-8f33-4bb2-b622-1dae9144a9fe-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-q2pns\" (UID: \"5cf90653-8f33-4bb2-b622-1dae9144a9fe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q2pns"
Oct 30 21:37:28.350008 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:28.349999    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:37:28.350214 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:28.350018    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a099b798-61cd-4306-85ba-85c7deb25a18-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-5lpq9\" (UID: \"a099b798-61cd-4306-85ba-85c7deb25a18\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-5lpq9"
Oct 30 21:37:28.350214 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:28.350044    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0535562-96d7-4fc4-a5d5-ad4cc785080e-metrics-certs\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:28.350214 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:28.350020    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 21:37:28.350214 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:28.350089    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55fbc4d8d7-w4mrq: secret "image-registry-tls" not found
Oct 30 21:37:28.350214 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:28.350098    2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 21:37:28.350214 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:28.350108    2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 21:37:28.350214 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:28.350110    2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 21:37:28.350214 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:28.350126    2568 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 21:37:28.350214 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:28.350066    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/872bccf3-b52e-4c52-9f5e-0487b4957b50-metrics-tls podName:872bccf3-b52e-4c52-9f5e-0487b4957b50 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:30.350044013 +0000 UTC m=+34.676536561 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/872bccf3-b52e-4c52-9f5e-0487b4957b50-metrics-tls") pod "dns-default-6q6pm" (UID: "872bccf3-b52e-4c52-9f5e-0487b4957b50") : secret "dns-default-metrics-tls" not found
Oct 30 21:37:28.350214 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:28.350173    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-registry-tls podName:0dca73b7-6526-48ac-a76e-70e441debc00 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:30.350157267 +0000 UTC m=+34.676649803 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-registry-tls") pod "image-registry-55fbc4d8d7-w4mrq" (UID: "0dca73b7-6526-48ac-a76e-70e441debc00") : secret "image-registry-tls" not found
Oct 30 21:37:28.350214 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:28.350195    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0535562-96d7-4fc4-a5d5-ad4cc785080e-metrics-certs podName:e0535562-96d7-4fc4-a5d5-ad4cc785080e nodeName:}" failed. No retries permitted until 2026-10-30 21:37:30.350185695 +0000 UTC m=+34.676678235 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e0535562-96d7-4fc4-a5d5-ad4cc785080e-metrics-certs") pod "router-default-57f669cfc9-d755h" (UID: "e0535562-96d7-4fc4-a5d5-ad4cc785080e") : secret "router-metrics-certs-default" not found
Oct 30 21:37:28.350214 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:28.350221    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65827025-dc76-42d8-be9f-bdfed6b28429-cluster-monitoring-operator-tls podName:65827025-dc76-42d8-be9f-bdfed6b28429 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:30.35021473 +0000 UTC m=+34.676707261 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/65827025-dc76-42d8-be9f-bdfed6b28429-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-wltlg" (UID: "65827025-dc76-42d8-be9f-bdfed6b28429") : secret "cluster-monitoring-operator-tls" not found
Oct 30 21:37:28.350648 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:28.350254    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cf90653-8f33-4bb2-b622-1dae9144a9fe-samples-operator-tls podName:5cf90653-8f33-4bb2-b622-1dae9144a9fe nodeName:}" failed. No retries permitted until 2026-10-30 21:37:30.350227762 +0000 UTC m=+34.676720294 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5cf90653-8f33-4bb2-b622-1dae9144a9fe-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-q2pns" (UID: "5cf90653-8f33-4bb2-b622-1dae9144a9fe") : secret "samples-operator-tls" not found
Oct 30 21:37:28.350648 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:28.350266    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a099b798-61cd-4306-85ba-85c7deb25a18-networking-console-plugin-cert podName:a099b798-61cd-4306-85ba-85c7deb25a18 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:30.350260552 +0000 UTC m=+34.676753085 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a099b798-61cd-4306-85ba-85c7deb25a18-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-5lpq9" (UID: "a099b798-61cd-4306-85ba-85c7deb25a18") : secret "networking-console-plugin-cert" not found
Oct 30 21:37:28.350648 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:28.350290    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-tls\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:28.350648 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:28.350367    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 21:37:28.350648 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:28.350375    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d487d746-hf6pr: secret "image-registry-tls" not found
Oct 30 21:37:28.350648 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:28.350405    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-tls podName:30931e26-479e-4ddb-8ce3-9b2b52dc0e8e nodeName:}" failed. No retries permitted until 2026-10-30 21:37:30.350396147 +0000 UTC m=+34.676888692 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-tls") pod "image-registry-7d487d746-hf6pr" (UID: "30931e26-479e-4ddb-8ce3-9b2b52dc0e8e") : secret "image-registry-tls" not found
Oct 30 21:37:28.416864 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:28.416812    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-px9dx" event={"ID":"5f5757a0-0a11-4637-9aa5-4e283792017d","Type":"ContainerStarted","Data":"62161da32f808f30211eda9c791d65fd22f9c71c4cc85e40f981b8f9ea770e97"}
Oct 30 21:37:28.418070 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:28.418043    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-58v4l" event={"ID":"51e670da-3db6-412b-8753-8606d7a20fe3","Type":"ContainerStarted","Data":"8141d1060aedcae9e8276e2435ae0d51406506284af41b835416efde5b1f99fa"}
Oct 30 21:37:28.419448 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:28.419426    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-nvwdl" event={"ID":"d54af961-c593-4707-9cec-1069000bce50","Type":"ContainerStarted","Data":"108a908739133f568953fc87a0c9e209beaf3a8146e4b1722eb0599d559fe279"}
Oct 30 21:37:28.420803 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:28.420776    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-tlfjh" event={"ID":"1dac48d8-aefb-4ff4-9213-b13def641c00","Type":"ContainerStarted","Data":"c688bc6120ebceeb7e5754b7f8642d7dad0c71664977c0691b07627ee903876c"}
Oct 30 21:37:28.420897 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:28.420817    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-tlfjh" event={"ID":"1dac48d8-aefb-4ff4-9213-b13def641c00","Type":"ContainerStarted","Data":"047c43343c8d3a6ca7dbb44be20d27b13bc9d3a62d2c27732b32c88271a79021"}
Oct 30 21:37:28.585433 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:28.585326    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-px9dx" podStartSLOduration=391.58530269 podStartE2EDuration="6m31.58530269s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:30:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:28.585159177 +0000 UTC m=+32.911651729" watchObservedRunningTime="2026-10-30 21:37:28.58530269 +0000 UTC m=+32.911795245"
Oct 30 21:37:28.694743 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:28.694678    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-nvwdl" podStartSLOduration=391.694658504 podStartE2EDuration="6m31.694658504s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:30:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:28.693080024 +0000 UTC m=+33.019572593" watchObservedRunningTime="2026-10-30 21:37:28.694658504 +0000 UTC m=+33.021151061"
Oct 30 21:37:29.974176 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:29.974135    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e615be3-41d7-4f82-90d7-7ce71ed9190b-original-pull-secret\") pod \"global-pull-secret-syncer-z9h2r\" (UID: \"6e615be3-41d7-4f82-90d7-7ce71ed9190b\") " pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:37:29.974792 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:29.974283    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-metrics-certs\") pod \"network-metrics-daemon-l9kxl\" (UID: \"7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2\") " pod="openshift-multus/network-metrics-daemon-l9kxl"
Oct 30 21:37:29.974792 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:29.974594    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 21:37:29.974792 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:29.974675    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-metrics-certs podName:7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2 nodeName:}" failed. No retries permitted until 2026-10-30 21:38:01.974654632 +0000 UTC m=+66.301147166 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-metrics-certs") pod "network-metrics-daemon-l9kxl" (UID: "7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2") : secret "metrics-daemon-secret" not found
Oct 30 21:37:29.977187 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:29.977155    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e615be3-41d7-4f82-90d7-7ce71ed9190b-original-pull-secret\") pod \"global-pull-secret-syncer-z9h2r\" (UID: \"6e615be3-41d7-4f82-90d7-7ce71ed9190b\") " pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:37:30.075258 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:30.075201    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxmv7\" (UniqueName: \"kubernetes.io/projected/9eefa84c-5304-4c2a-89cf-3304097ecf42-kube-api-access-gxmv7\") pod \"network-check-target-wpc6f\" (UID: \"9eefa84c-5304-4c2a-89cf-3304097ecf42\") " pod="openshift-network-diagnostics/network-check-target-wpc6f"
Oct 30 21:37:30.084048 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:30.084016    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxmv7\" (UniqueName: \"kubernetes.io/projected/9eefa84c-5304-4c2a-89cf-3304097ecf42-kube-api-access-gxmv7\") pod \"network-check-target-wpc6f\" (UID: \"9eefa84c-5304-4c2a-89cf-3304097ecf42\") " pod="openshift-network-diagnostics/network-check-target-wpc6f"
Oct 30 21:37:30.101692 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:30.101650    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:37:30.277713 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:30.277009    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0535562-96d7-4fc4-a5d5-ad4cc785080e-service-ca-bundle\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:30.277713 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:30.277262    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e0535562-96d7-4fc4-a5d5-ad4cc785080e-service-ca-bundle podName:e0535562-96d7-4fc4-a5d5-ad4cc785080e nodeName:}" failed. No retries permitted until 2026-10-30 21:37:34.277223025 +0000 UTC m=+38.603715565 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e0535562-96d7-4fc4-a5d5-ad4cc785080e-service-ca-bundle") pod "router-default-57f669cfc9-d755h" (UID: "e0535562-96d7-4fc4-a5d5-ad4cc785080e") : configmap references non-existent config key: service-ca.crt
Oct 30 21:37:30.378082 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:30.378043    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-registry-tls\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:30.378354 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:30.378219    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 21:37:30.378354 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:30.378263    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55fbc4d8d7-w4mrq: secret "image-registry-tls" not found
Oct 30 21:37:30.378354 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:30.378270    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/872bccf3-b52e-4c52-9f5e-0487b4957b50-metrics-tls\") pod \"dns-default-6q6pm\" (UID: \"872bccf3-b52e-4c52-9f5e-0487b4957b50\") " pod="openshift-dns/dns-default-6q6pm"
Oct 30 21:37:30.378354 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:30.378324    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/65827025-dc76-42d8-be9f-bdfed6b28429-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-wltlg\" (UID: \"65827025-dc76-42d8-be9f-bdfed6b28429\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wltlg"
Oct 30 21:37:30.378602 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:30.378369    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-registry-tls podName:0dca73b7-6526-48ac-a76e-70e441debc00 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:34.378348084 +0000 UTC m=+38.704840623 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-registry-tls") pod "image-registry-55fbc4d8d7-w4mrq" (UID: "0dca73b7-6526-48ac-a76e-70e441debc00") : secret "image-registry-tls" not found
Oct 30 21:37:30.378602 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:30.378425    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5cf90653-8f33-4bb2-b622-1dae9144a9fe-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-q2pns\" (UID: \"5cf90653-8f33-4bb2-b622-1dae9144a9fe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q2pns"
Oct 30 21:37:30.378602 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:30.378463    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a099b798-61cd-4306-85ba-85c7deb25a18-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-5lpq9\" (UID: \"a099b798-61cd-4306-85ba-85c7deb25a18\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-5lpq9"
Oct 30 21:37:30.378602 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:30.378495    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0535562-96d7-4fc4-a5d5-ad4cc785080e-metrics-certs\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:30.378602 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:30.378529    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-tls\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:30.378891 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:30.378429    2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 21:37:30.378891 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:30.378722    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65827025-dc76-42d8-be9f-bdfed6b28429-cluster-monitoring-operator-tls podName:65827025-dc76-42d8-be9f-bdfed6b28429 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:34.378706675 +0000 UTC m=+38.705199211 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/65827025-dc76-42d8-be9f-bdfed6b28429-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-wltlg" (UID: "65827025-dc76-42d8-be9f-bdfed6b28429") : secret "cluster-monitoring-operator-tls" not found
Oct 30 21:37:30.378891 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:30.378779    2568 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 21:37:30.378891 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:30.378810    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a099b798-61cd-4306-85ba-85c7deb25a18-networking-console-plugin-cert podName:a099b798-61cd-4306-85ba-85c7deb25a18 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:34.378799392 +0000 UTC m=+38.705291925 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a099b798-61cd-4306-85ba-85c7deb25a18-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-5lpq9" (UID: "a099b798-61cd-4306-85ba-85c7deb25a18") : secret "networking-console-plugin-cert" not found
Oct 30 21:37:30.378891 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:30.378858    2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 21:37:30.378891 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:30.378889    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0535562-96d7-4fc4-a5d5-ad4cc785080e-metrics-certs podName:e0535562-96d7-4fc4-a5d5-ad4cc785080e nodeName:}" failed. No retries permitted until 2026-10-30 21:37:34.378878996 +0000 UTC m=+38.705371538 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e0535562-96d7-4fc4-a5d5-ad4cc785080e-metrics-certs") pod "router-default-57f669cfc9-d755h" (UID: "e0535562-96d7-4fc4-a5d5-ad4cc785080e") : secret "router-metrics-certs-default" not found
Oct 30 21:37:30.378891 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:30.378489    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:37:30.379316 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:30.378921    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/872bccf3-b52e-4c52-9f5e-0487b4957b50-metrics-tls podName:872bccf3-b52e-4c52-9f5e-0487b4957b50 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:34.378911839 +0000 UTC m=+38.705404376 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/872bccf3-b52e-4c52-9f5e-0487b4957b50-metrics-tls") pod "dns-default-6q6pm" (UID: "872bccf3-b52e-4c52-9f5e-0487b4957b50") : secret "dns-default-metrics-tls" not found
Oct 30 21:37:30.379316 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:30.378540    2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 21:37:30.379316 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:30.378954    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cf90653-8f33-4bb2-b622-1dae9144a9fe-samples-operator-tls podName:5cf90653-8f33-4bb2-b622-1dae9144a9fe nodeName:}" failed. No retries permitted until 2026-10-30 21:37:34.378945246 +0000 UTC m=+38.705437780 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5cf90653-8f33-4bb2-b622-1dae9144a9fe-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-q2pns" (UID: "5cf90653-8f33-4bb2-b622-1dae9144a9fe") : secret "samples-operator-tls" not found
Oct 30 21:37:30.379316 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:30.378670    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 21:37:30.379316 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:30.378975    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d487d746-hf6pr: secret "image-registry-tls" not found
Oct 30 21:37:30.379316 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:30.379031    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-tls podName:30931e26-479e-4ddb-8ce3-9b2b52dc0e8e nodeName:}" failed. No retries permitted until 2026-10-30 21:37:34.378992661 +0000 UTC m=+38.705485196 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-tls") pod "image-registry-7d487d746-hf6pr" (UID: "30931e26-479e-4ddb-8ce3-9b2b52dc0e8e") : secret "image-registry-tls" not found
Oct 30 21:37:30.382229 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:30.382199    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wpc6f"
Oct 30 21:37:30.572844 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:30.570769    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-tlfjh" podStartSLOduration=393.57074911 podStartE2EDuration="6m33.57074911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:30:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:28.804345657 +0000 UTC m=+33.130838212" watchObservedRunningTime="2026-10-30 21:37:30.57074911 +0000 UTC m=+34.897241668"
Oct 30 21:37:30.572844 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:30.572122    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-z9h2r"]
Oct 30 21:37:30.582906 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:37:30.582864    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e615be3_41d7_4f82_90d7_7ce71ed9190b.slice/crio-1d384fd2f42fcd3364665cf5490cfcfb2ffe9be5cc6174f2dafcb6f37d6cb55c WatchSource:0}: Error finding container 1d384fd2f42fcd3364665cf5490cfcfb2ffe9be5cc6174f2dafcb6f37d6cb55c: Status 404 returned error can't find the container with id 1d384fd2f42fcd3364665cf5490cfcfb2ffe9be5cc6174f2dafcb6f37d6cb55c
Oct 30 21:37:30.610917 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:30.610852    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:30.818381 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:30.818268    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wpc6f"]
Oct 30 21:37:30.827715 ip-10-0-139-46 kubenswrapper[2568]: W1030 21:37:30.827679    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eefa84c_5304_4c2a_89cf_3304097ecf42.slice/crio-4f516aad48abe9950750e12d96a48abf731a56b0e54c1424bd180419fc148b9c WatchSource:0}: Error finding container 4f516aad48abe9950750e12d96a48abf731a56b0e54c1424bd180419fc148b9c: Status 404 returned error can't find the container with id 4f516aad48abe9950750e12d96a48abf731a56b0e54c1424bd180419fc148b9c
Oct 30 21:37:30.830927 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:30.830396    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:31.441383 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:31.440413    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-7lnsg" event={"ID":"32888534-d682-4428-90d9-03c858ce5ab8","Type":"ContainerStarted","Data":"67135c0a7c4518e5b2fb61d7c1d493991c6d9756d0db0c12b30ced734baa4430"}
Oct 30 21:37:31.444094 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:31.444040    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wpc6f" event={"ID":"9eefa84c-5304-4c2a-89cf-3304097ecf42","Type":"ContainerStarted","Data":"4f516aad48abe9950750e12d96a48abf731a56b0e54c1424bd180419fc148b9c"}
Oct 30 21:37:31.445641 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:31.445615    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-z9h2r" event={"ID":"6e615be3-41d7-4f82-90d7-7ce71ed9190b","Type":"ContainerStarted","Data":"1d384fd2f42fcd3364665cf5490cfcfb2ffe9be5cc6174f2dafcb6f37d6cb55c"}
Oct 30 21:37:31.550957 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:31.550880    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-7lnsg" podStartSLOduration=381.108767912 podStartE2EDuration="6m24.550861141s" totalImagesPullingTime="3.442093229s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:31:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:31.550533483 +0000 UTC m=+35.877026051" watchObservedRunningTime="2026-10-30 21:37:31.550861141 +0000 UTC m=+35.877353709"
Oct 30 21:37:31.744621 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:31.744533    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:33.467435 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:33.467136    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-pf9px" event={"ID":"4086d1bd-6e47-4d50-a7f9-5be23d123d6e","Type":"ContainerStarted","Data":"e01e645166be176533871d4d4a0a2e62a088ea447d78c75b822e28d17e7b8afd"}
Oct 30 21:37:33.482057 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:33.481550    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-qqxdq" event={"ID":"fb70ad86-4b45-4a53-a3cb-1f6b59640708","Type":"ContainerStarted","Data":"314e80b5158e2c5213dc954620b41d97abef4c7368175cb60a52ab81231ed958"}
Oct 30 21:37:33.486679 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:33.486135    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-jgspb" event={"ID":"37186e06-8d56-4c61-ba10-e0408d2e2fa4","Type":"ContainerStarted","Data":"450b76fbd57f402c2f415a74e460d3495cbf9fa7a9222b7ab243b8e03e41e067"}
Oct 30 21:37:33.494051 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:33.491588    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f" event={"ID":"b7002d6d-834c-4bdc-907e-805edd577a40","Type":"ContainerStarted","Data":"ec953196f535ba82443c855bc6901e2f6f9701068e29b3245b8b869eedd0ccac"}
Oct 30 21:37:33.494051 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:33.493862    2568 patch_prober.go:29] interesting pod/console-operator-7cd56b8dbc-7tf9f container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.133.0.17:8443/readyz\": dial tcp 10.133.0.17:8443: connect: connection refused" start-of-body=
Oct 30 21:37:33.494051 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:33.493913    2568 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f" podUID="b7002d6d-834c-4bdc-907e-805edd577a40" containerName="console-operator" probeResult="failure" output="Get \"https://10.133.0.17:8443/readyz\": dial tcp 10.133.0.17:8443: connect: connection refused"
Oct 30 21:37:33.494051 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:33.493996    2568 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f"
Oct 30 21:37:33.583181 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:33.582355    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-operator-7c4986d449-pf9px" podStartSLOduration=378.39277157 podStartE2EDuration="6m23.582335763s" totalImagesPullingTime="5.189564193s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:31:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:33.581641787 +0000 UTC m=+37.908134341" watchObservedRunningTime="2026-10-30 21:37:33.582335763 +0000 UTC m=+37.908828321"
Oct 30 21:37:33.688338 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:33.688305    2568 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-h6hzg"]
Oct 30 21:37:33.695668 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:33.695606    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f" podStartSLOduration=380.797682759 podStartE2EDuration="6m26.695586904s" totalImagesPullingTime="5.897904145s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:31:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:33.69471895 +0000 UTC m=+38.021211504" watchObservedRunningTime="2026-10-30 21:37:33.695586904 +0000 UTC m=+38.022079461"
Oct 30 21:37:33.706114 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:33.706080    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:33.720296 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:33.720268    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:33.720418 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:33.720329    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-crio-socket\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:33.720418 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:33.720395    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-data-volume\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:33.720531 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:33.720420    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:33.720531 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:33.720461    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zgrn\" (UniqueName: \"kubernetes.io/projected/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-kube-api-access-2zgrn\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:33.784091 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:33.780507    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-h6hzg"]
Oct 30 21:37:33.808398 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:33.808332    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-qqxdq" podStartSLOduration=385.858828803 podStartE2EDuration="6m31.808311441s" totalImagesPullingTime="5.949482638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:31:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:33.806720806 +0000 UTC m=+38.133213365" watchObservedRunningTime="2026-10-30 21:37:33.808311441 +0000 UTC m=+38.134803997"
Oct 30 21:37:33.820971 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:33.820921    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-data-volume\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:33.821106 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:33.820974    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:33.821106 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:33.821044    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zgrn\" (UniqueName: \"kubernetes.io/projected/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-kube-api-access-2zgrn\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:33.821306 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:33.821115    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:33.821306 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:33.821178    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-crio-socket\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:33.821456 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:33.821312    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-crio-socket\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:33.822405 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:33.821622    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-data-volume\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:33.828866 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:33.828823    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 21:37:33.829564 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:33.829511    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xlcqp\""
Oct 30 21:37:33.832558 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:33.832307    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 21:37:33.832558 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:33.832372    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 21:37:33.832558 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:33.832445    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-kube-rbac-proxy-cm podName:565e807a-a9c3-4f1b-8de0-9ff4d713a1ee nodeName:}" failed. No retries permitted until 2026-10-30 21:37:34.332424846 +0000 UTC m=+38.658917380 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-kube-rbac-proxy-cm") pod "insights-runtime-extractor-h6hzg" (UID: "565e807a-a9c3-4f1b-8de0-9ff4d713a1ee") : configmap "kube-rbac-proxy" not found
Oct 30 21:37:33.832558 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:33.832477    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:37:33.832558 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:33.832513    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-insights-runtime-extractor-tls podName:565e807a-a9c3-4f1b-8de0-9ff4d713a1ee nodeName:}" failed. No retries permitted until 2026-10-30 21:37:34.332502891 +0000 UTC m=+38.658995429 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-insights-runtime-extractor-tls") pod "insights-runtime-extractor-h6hzg" (UID: "565e807a-a9c3-4f1b-8de0-9ff4d713a1ee") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:37:33.895259 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:33.888222    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zgrn\" (UniqueName: \"kubernetes.io/projected/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-kube-api-access-2zgrn\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:33.932497 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:33.932457    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:34.035038 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:34.034972    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-jgspb" podStartSLOduration=380.042372166 podStartE2EDuration="6m26.034948393s" totalImagesPullingTime="5.992576227s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:31:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:33.922185098 +0000 UTC m=+38.248677653" watchObservedRunningTime="2026-10-30 21:37:34.034948393 +0000 UTC m=+38.361440948"
Oct 30 21:37:34.146224 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:34.146148    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:34.326870 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:34.326832    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0535562-96d7-4fc4-a5d5-ad4cc785080e-service-ca-bundle\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:34.327041 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:34.327021    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e0535562-96d7-4fc4-a5d5-ad4cc785080e-service-ca-bundle podName:e0535562-96d7-4fc4-a5d5-ad4cc785080e nodeName:}" failed. No retries permitted until 2026-10-30 21:37:42.326996765 +0000 UTC m=+46.653489302 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e0535562-96d7-4fc4-a5d5-ad4cc785080e-service-ca-bundle") pod "router-default-57f669cfc9-d755h" (UID: "e0535562-96d7-4fc4-a5d5-ad4cc785080e") : configmap references non-existent config key: service-ca.crt
Oct 30 21:37:34.427796 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:34.427753    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5cf90653-8f33-4bb2-b622-1dae9144a9fe-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-q2pns\" (UID: \"5cf90653-8f33-4bb2-b622-1dae9144a9fe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q2pns"
Oct 30 21:37:34.428038 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:34.427807    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:34.428038 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:34.427845    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a099b798-61cd-4306-85ba-85c7deb25a18-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-5lpq9\" (UID: \"a099b798-61cd-4306-85ba-85c7deb25a18\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-5lpq9"
Oct 30 21:37:34.428038 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:34.427875    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0535562-96d7-4fc4-a5d5-ad4cc785080e-metrics-certs\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:34.428038 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:34.427900    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-tls\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:34.428038 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:34.427902    2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 21:37:34.428038 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:34.427944    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-registry-tls\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:34.428038 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:34.427969    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cf90653-8f33-4bb2-b622-1dae9144a9fe-samples-operator-tls podName:5cf90653-8f33-4bb2-b622-1dae9144a9fe nodeName:}" failed. No retries permitted until 2026-10-30 21:37:42.427953561 +0000 UTC m=+46.754446112 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5cf90653-8f33-4bb2-b622-1dae9144a9fe-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-q2pns" (UID: "5cf90653-8f33-4bb2-b622-1dae9144a9fe") : secret "samples-operator-tls" not found
Oct 30 21:37:34.428038 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:34.428025    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:34.428038 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:34.428041    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 21:37:34.428523 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:34.428055    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55fbc4d8d7-w4mrq: secret "image-registry-tls" not found
Oct 30 21:37:34.428523 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:34.428058    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/872bccf3-b52e-4c52-9f5e-0487b4957b50-metrics-tls\") pod \"dns-default-6q6pm\" (UID: \"872bccf3-b52e-4c52-9f5e-0487b4957b50\") " pod="openshift-dns/dns-default-6q6pm"
Oct 30 21:37:34.428523 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:34.428086    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/65827025-dc76-42d8-be9f-bdfed6b28429-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-wltlg\" (UID: \"65827025-dc76-42d8-be9f-bdfed6b28429\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wltlg"
Oct 30 21:37:34.428523 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:34.428101    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-registry-tls podName:0dca73b7-6526-48ac-a76e-70e441debc00 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:42.428086412 +0000 UTC m=+46.754578949 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-registry-tls") pod "image-registry-55fbc4d8d7-w4mrq" (UID: "0dca73b7-6526-48ac-a76e-70e441debc00") : secret "image-registry-tls" not found
Oct 30 21:37:34.428523 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:34.428134    2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 21:37:34.428523 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:34.428160    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65827025-dc76-42d8-be9f-bdfed6b28429-cluster-monitoring-operator-tls podName:65827025-dc76-42d8-be9f-bdfed6b28429 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:42.428152186 +0000 UTC m=+46.754644719 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/65827025-dc76-42d8-be9f-bdfed6b28429-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-wltlg" (UID: "65827025-dc76-42d8-be9f-bdfed6b28429") : secret "cluster-monitoring-operator-tls" not found
Oct 30 21:37:34.428523 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:34.428205    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 21:37:34.428523 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:34.428226    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-kube-rbac-proxy-cm podName:565e807a-a9c3-4f1b-8de0-9ff4d713a1ee nodeName:}" failed. No retries permitted until 2026-10-30 21:37:35.428220785 +0000 UTC m=+39.754713318 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-kube-rbac-proxy-cm") pod "insights-runtime-extractor-h6hzg" (UID: "565e807a-a9c3-4f1b-8de0-9ff4d713a1ee") : configmap "kube-rbac-proxy" not found
Oct 30 21:37:34.428523 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:34.428230    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:37:34.428523 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:34.428281    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:37:34.428523 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:34.428300    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/872bccf3-b52e-4c52-9f5e-0487b4957b50-metrics-tls podName:872bccf3-b52e-4c52-9f5e-0487b4957b50 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:42.428294011 +0000 UTC m=+46.754786543 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/872bccf3-b52e-4c52-9f5e-0487b4957b50-metrics-tls") pod "dns-default-6q6pm" (UID: "872bccf3-b52e-4c52-9f5e-0487b4957b50") : secret "dns-default-metrics-tls" not found
Oct 30 21:37:34.428523 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:34.428311    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-insights-runtime-extractor-tls podName:565e807a-a9c3-4f1b-8de0-9ff4d713a1ee nodeName:}" failed. No retries permitted until 2026-10-30 21:37:35.428305532 +0000 UTC m=+39.754798064 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-insights-runtime-extractor-tls") pod "insights-runtime-extractor-h6hzg" (UID: "565e807a-a9c3-4f1b-8de0-9ff4d713a1ee") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:37:34.428523 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:34.428351    2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 21:37:34.428523 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:34.428354    2568 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 21:37:34.428523 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:34.428370    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0535562-96d7-4fc4-a5d5-ad4cc785080e-metrics-certs podName:e0535562-96d7-4fc4-a5d5-ad4cc785080e nodeName:}" failed. No retries permitted until 2026-10-30 21:37:42.428364124 +0000 UTC m=+46.754856699 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e0535562-96d7-4fc4-a5d5-ad4cc785080e-metrics-certs") pod "router-default-57f669cfc9-d755h" (UID: "e0535562-96d7-4fc4-a5d5-ad4cc785080e") : secret "router-metrics-certs-default" not found
Oct 30 21:37:34.429159 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:34.428389    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a099b798-61cd-4306-85ba-85c7deb25a18-networking-console-plugin-cert podName:a099b798-61cd-4306-85ba-85c7deb25a18 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:42.428376663 +0000 UTC m=+46.754869201 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a099b798-61cd-4306-85ba-85c7deb25a18-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-5lpq9" (UID: "a099b798-61cd-4306-85ba-85c7deb25a18") : secret "networking-console-plugin-cert" not found
Oct 30 21:37:34.429159 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:34.428408    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 21:37:34.429159 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:34.428415    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d487d746-hf6pr: secret "image-registry-tls" not found
Oct 30 21:37:34.429159 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:34.428433    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-tls podName:30931e26-479e-4ddb-8ce3-9b2b52dc0e8e nodeName:}" failed. No retries permitted until 2026-10-30 21:37:42.428428077 +0000 UTC m=+46.754920662 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-tls") pod "image-registry-7d487d746-hf6pr" (UID: "30931e26-479e-4ddb-8ce3-9b2b52dc0e8e") : secret "image-registry-tls" not found
Oct 30 21:37:34.497680 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:34.497654    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/0.log"
Oct 30 21:37:34.498072 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:34.497704    2568 generic.go:356] "Generic (PLEG): container finished" podID="b7002d6d-834c-4bdc-907e-805edd577a40" containerID="ec953196f535ba82443c855bc6901e2f6f9701068e29b3245b8b869eedd0ccac" exitCode=255
Oct 30 21:37:34.498072 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:34.497795    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f" event={"ID":"b7002d6d-834c-4bdc-907e-805edd577a40","Type":"ContainerDied","Data":"ec953196f535ba82443c855bc6901e2f6f9701068e29b3245b8b869eedd0ccac"}
Oct 30 21:37:34.498072 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:34.498062    2568 scope.go:122] "RemoveContainer" containerID="ec953196f535ba82443c855bc6901e2f6f9701068e29b3245b8b869eedd0ccac"
Oct 30 21:37:34.503076 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:34.503006    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rqxfc" event={"ID":"be4fa9bf-41e0-4b26-9457-f9815a36d5b5","Type":"ContainerStarted","Data":"c8285dfeb4ee5170c0b8451aa477567726efe830e4a381ba548e4c487a5aef2a"}
Oct 30 21:37:34.719409 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:34.719341    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rqxfc" podStartSLOduration=2.2909057170000002 podStartE2EDuration="8.719319552s" totalImagesPullingTime="6.428413835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:37:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:34.718020039 +0000 UTC m=+39.044512592" watchObservedRunningTime="2026-10-30 21:37:34.719319552 +0000 UTC m=+39.045812108"
Oct 30 21:37:34.773966 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:34.773930    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:34.939063 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:34.939015    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:35.116303 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:35.116263    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:35.300412 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:35.300328    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:35.442368 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:35.442123    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:35.442368 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:35.442276    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 21:37:35.442368 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:35.442288    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:35.442368 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:35.442367    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-kube-rbac-proxy-cm podName:565e807a-a9c3-4f1b-8de0-9ff4d713a1ee nodeName:}" failed. No retries permitted until 2026-10-30 21:37:37.442350478 +0000 UTC m=+41.768843015 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-kube-rbac-proxy-cm") pod "insights-runtime-extractor-h6hzg" (UID: "565e807a-a9c3-4f1b-8de0-9ff4d713a1ee") : configmap "kube-rbac-proxy" not found
Oct 30 21:37:35.442647 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:35.442433    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:37:35.442647 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:35.442493    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-insights-runtime-extractor-tls podName:565e807a-a9c3-4f1b-8de0-9ff4d713a1ee nodeName:}" failed. No retries permitted until 2026-10-30 21:37:37.442478642 +0000 UTC m=+41.768971188 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-insights-runtime-extractor-tls") pod "insights-runtime-extractor-h6hzg" (UID: "565e807a-a9c3-4f1b-8de0-9ff4d713a1ee") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:37:35.491625 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:35.491576    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:35.514488 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:35.514457    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/1.log"
Oct 30 21:37:35.514963 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:35.514946    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/0.log"
Oct 30 21:37:35.515019 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:35.514982    2568 generic.go:356] "Generic (PLEG): container finished" podID="b7002d6d-834c-4bdc-907e-805edd577a40" containerID="b357894139be464ca4c0abc3e8e4b114e23bf2f0cc6df8213aaa1e84a3a6e451" exitCode=255
Oct 30 21:37:35.515068 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:35.515054    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f" event={"ID":"b7002d6d-834c-4bdc-907e-805edd577a40","Type":"ContainerDied","Data":"b357894139be464ca4c0abc3e8e4b114e23bf2f0cc6df8213aaa1e84a3a6e451"}
Oct 30 21:37:35.515145 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:35.515131    2568 scope.go:122] "RemoveContainer" containerID="ec953196f535ba82443c855bc6901e2f6f9701068e29b3245b8b869eedd0ccac"
Oct 30 21:37:35.515364 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:35.515343    2568 scope.go:122] "RemoveContainer" containerID="b357894139be464ca4c0abc3e8e4b114e23bf2f0cc6df8213aaa1e84a3a6e451"
Oct 30 21:37:35.515621 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:35.515594    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-7tf9f_openshift-console-operator(b7002d6d-834c-4bdc-907e-805edd577a40)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f" podUID="b7002d6d-834c-4bdc-907e-805edd577a40"
Oct 30 21:37:35.517232 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:35.517205    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-58v4l" event={"ID":"51e670da-3db6-412b-8753-8606d7a20fe3","Type":"ContainerStarted","Data":"f129905556fb283caa63f037f1154b341ab9686612701c7ee49c6c7eea53e24a"}
Oct 30 21:37:35.518941 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:35.518908    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wpc6f" event={"ID":"9eefa84c-5304-4c2a-89cf-3304097ecf42","Type":"ContainerStarted","Data":"d155037c0f10d10c13fe52429a9084538dacfaba640f5cc7b44610e8ad28cbc1"}
Oct 30 21:37:35.646684 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:35.646564    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:35.745102 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:35.745036    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-wpc6f" podStartSLOduration=35.681174925 podStartE2EDuration="39.745019255s" totalImagesPullingTime="4.06384433s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:36:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:35.743554702 +0000 UTC m=+40.070047285" watchObservedRunningTime="2026-10-30 21:37:35.745019255 +0000 UTC m=+40.071511809"
Oct 30 21:37:35.858026 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:35.857960    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-58v4l" podStartSLOduration=342.885439548 podStartE2EDuration="5m49.857937463s" totalImagesPullingTime="6.972497915s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:31:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:35.856106172 +0000 UTC m=+40.182598727" watchObservedRunningTime="2026-10-30 21:37:35.857937463 +0000 UTC m=+40.184430029"
Oct 30 21:37:36.233725 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:36.233686    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:36.472469 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:36.472426    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:36.524098 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:36.524027    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/1.log"
Oct 30 21:37:36.524568 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:36.524443    2568 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-wpc6f"
Oct 30 21:37:36.524568 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:36.524448    2568 scope.go:122] "RemoveContainer" containerID="b357894139be464ca4c0abc3e8e4b114e23bf2f0cc6df8213aaa1e84a3a6e451"
Oct 30 21:37:36.524686 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:36.524665    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-7tf9f_openshift-console-operator(b7002d6d-834c-4bdc-907e-805edd577a40)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f" podUID="b7002d6d-834c-4bdc-907e-805edd577a40"
Oct 30 21:37:36.967090 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:36.967007    2568 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f"
Oct 30 21:37:37.281490 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:37.281405    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:37.463770 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:37.463728    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:37.463875 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:37.463858    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 21:37:37.463939 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:37.463926    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-kube-rbac-proxy-cm podName:565e807a-a9c3-4f1b-8de0-9ff4d713a1ee nodeName:}" failed. No retries permitted until 2026-10-30 21:37:41.463909594 +0000 UTC m=+45.790402147 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-kube-rbac-proxy-cm") pod "insights-runtime-extractor-h6hzg" (UID: "565e807a-a9c3-4f1b-8de0-9ff4d713a1ee") : configmap "kube-rbac-proxy" not found
Oct 30 21:37:37.463990 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:37.463945    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:37.464073 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:37.464061    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:37:37.464115 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:37.464095    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-insights-runtime-extractor-tls podName:565e807a-a9c3-4f1b-8de0-9ff4d713a1ee nodeName:}" failed. No retries permitted until 2026-10-30 21:37:41.464087269 +0000 UTC m=+45.790579813 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-insights-runtime-extractor-tls") pod "insights-runtime-extractor-h6hzg" (UID: "565e807a-a9c3-4f1b-8de0-9ff4d713a1ee") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:37:37.528851 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:37.528821    2568 scope.go:122] "RemoveContainer" containerID="b357894139be464ca4c0abc3e8e4b114e23bf2f0cc6df8213aaa1e84a3a6e451"
Oct 30 21:37:37.529203 ip-10-0-139-46 kubenswrapper[2568]: E1030 21:37:37.528977    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-7tf9f_openshift-console-operator(b7002d6d-834c-4bdc-907e-805edd577a40)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f" podUID="b7002d6d-834c-4bdc-907e-805edd577a40"
Oct 30 21:37:37.760618 ip-10-0-139-46 kubenswrapper[2568]: I1030 21:37:37.760535    2568 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:37:37.760687 ip-10-0-139-46 systemd[1]: Stopping Kubernetes Kubelet...
Oct 30 21:37:37.770948 ip-10-0-139-46 systemd[1]: kubelet.service: Deactivated successfully.
Oct 30 21:37:37.771122 ip-10-0-139-46 systemd[1]: Stopped Kubernetes Kubelet.
Oct 30 21:37:37.771180 ip-10-0-139-46 systemd[1]: kubelet.service: Consumed 3.146s CPU time.
Oct 30 21:37:37.790753 ip-10-0-139-46 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 21:37:37.875314 ip-10-0-139-46 kubenswrapper[4871]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:37:37.875314 ip-10-0-139-46 kubenswrapper[4871]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 21:37:37.875314 ip-10-0-139-46 kubenswrapper[4871]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:37:37.875314 ip-10-0-139-46 kubenswrapper[4871]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:37:37.877733 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877633    4871 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:37:37.877733 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877725    4871 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:37:37.877733 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877730    4871 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:37:37.877733 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877735    4871 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:37:37.877733 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877739    4871 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:37:37.877733 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877743    4871 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:37:37.878100 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877747    4871 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:37:37.878100 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877751    4871 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:37:37.878100 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877755    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:37:37.878100 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877760    4871 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:37:37.878100 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877764    4871 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:37:37.878100 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877767    4871 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:37:37.878100 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877771    4871 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:37:37.878100 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877774    4871 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:37:37.878100 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877778    4871 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:37:37.878100 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877782    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:37:37.878100 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877787    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:37:37.878100 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877792    4871 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:37:37.878100 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877797    4871 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:37:37.878100 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877801    4871 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:37:37.878100 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877805    4871 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:37:37.878100 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877809    4871 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:37:37.878100 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877813    4871 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:37:37.878100 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877817    4871 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:37:37.878100 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877822    4871 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:37:37.878962 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877825    4871 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:37:37.878962 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877829    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:37:37.878962 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877833    4871 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:37:37.878962 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877838    4871 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:37:37.878962 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877842    4871 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:37:37.878962 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877845    4871 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:37:37.878962 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877850    4871 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:37:37.878962 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877854    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:37:37.878962 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877857    4871 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:37:37.878962 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877861    4871 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:37:37.878962 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877865    4871 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:37:37.878962 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877869    4871 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:37:37.878962 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877873    4871 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:37:37.878962 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877877    4871 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:37:37.878962 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877881    4871 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:37:37.878962 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877887    4871 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:37:37.878962 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877892    4871 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:37:37.878962 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877897    4871 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:37:37.878962 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877901    4871 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:37:37.879809 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877905    4871 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:37:37.879809 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877910    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:37:37.879809 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877914    4871 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:37:37.879809 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877919    4871 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:37:37.879809 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877923    4871 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:37:37.879809 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877928    4871 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:37:37.879809 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877932    4871 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:37:37.879809 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877937    4871 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:37:37.879809 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877941    4871 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:37:37.879809 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877945    4871 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:37:37.879809 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877949    4871 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:37:37.879809 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877953    4871 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:37:37.879809 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877958    4871 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:37:37.879809 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877962    4871 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:37:37.879809 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877970    4871 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:37:37.879809 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877978    4871 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:37:37.879809 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877984    4871 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:37:37.879809 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877988    4871 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:37:37.879809 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877993    4871 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:37:37.880652 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.877997    4871 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:37:37.880652 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878001    4871 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:37:37.880652 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878006    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:37:37.880652 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878010    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:37:37.880652 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878014    4871 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:37:37.880652 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878018    4871 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:37:37.880652 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878023    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:37:37.880652 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878027    4871 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:37:37.880652 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878031    4871 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:37:37.880652 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878035    4871 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:37:37.880652 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878040    4871 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:37:37.880652 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878044    4871 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:37:37.880652 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878049    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:37:37.880652 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878055    4871 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:37:37.880652 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878062    4871 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:37:37.880652 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878071    4871 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:37:37.880652 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878076    4871 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:37:37.880652 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878081    4871 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:37:37.880652 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878086    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:37:37.881518 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878090    4871 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:37:37.881518 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878095    4871 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:37:37.881518 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878098    4871 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:37:37.881518 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878103    4871 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:37:37.881518 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878107    4871 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:37:37.881518 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878111    4871 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:37:37.881518 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878115    4871 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:37:37.881518 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878119    4871 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:37:37.881518 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878124    4871 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:37:37.881518 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878128    4871 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:37:37.881518 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878133    4871 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:37:37.881518 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878137    4871 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:37:37.881518 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878143    4871 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:37:37.881518 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878147    4871 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:37:37.881518 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878151    4871 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:37:37.881518 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878156    4871 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:37:37.881518 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878160    4871 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:37:37.881518 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878165    4871 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:37:37.881518 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878168    4871 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:37:37.881518 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878173    4871 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:37:37.882414 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878177    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:37:37.882414 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878181    4871 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:37:37.882414 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878185    4871 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:37:37.882414 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.878190    4871 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:37:37.882414 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879482    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:37:37.882414 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879493    4871 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:37:37.882414 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879498    4871 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:37:37.882414 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879503    4871 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:37:37.882414 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879509    4871 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:37:37.882414 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879514    4871 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:37:37.882414 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879519    4871 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:37:37.882414 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879524    4871 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:37:37.882414 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879528    4871 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:37:37.882414 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879533    4871 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:37:37.882414 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879537    4871 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:37:37.882414 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879541    4871 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:37:37.882414 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879546    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:37:37.882414 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879550    4871 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:37:37.882414 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879554    4871 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:37:37.883339 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879558    4871 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:37:37.883339 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879563    4871 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:37:37.883339 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879568    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:37:37.883339 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879572    4871 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:37:37.883339 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879576    4871 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:37:37.883339 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879580    4871 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:37:37.883339 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879585    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:37:37.883339 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879589    4871 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:37:37.883339 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879593    4871 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:37:37.883339 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879597    4871 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:37:37.883339 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879602    4871 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:37:37.883339 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879606    4871 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:37:37.883339 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879610    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:37:37.883339 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879615    4871 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:37:37.883339 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879619    4871 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:37:37.883339 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879623    4871 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:37:37.883339 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879628    4871 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:37:37.883339 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879632    4871 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:37:37.883339 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879637    4871 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:37:37.883339 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879642    4871 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:37:37.884173 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879647    4871 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:37:37.884173 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879651    4871 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:37:37.884173 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879655    4871 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:37:37.884173 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879661    4871 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:37:37.884173 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879665    4871 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:37:37.884173 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879669    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:37:37.884173 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879673    4871 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:37:37.884173 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879677    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:37:37.884173 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879681    4871 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:37:37.884173 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879686    4871 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:37:37.884173 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879690    4871 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:37:37.884173 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879694    4871 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:37:37.884173 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879698    4871 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:37:37.884173 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879703    4871 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:37:37.884173 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879707    4871 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:37:37.884173 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879711    4871 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:37:37.884173 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879715    4871 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:37:37.884173 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879721    4871 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:37:37.884173 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879725    4871 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:37:37.884173 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879729    4871 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:37:37.884734 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879734    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:37:37.884734 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879738    4871 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:37:37.884734 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879742    4871 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:37:37.884734 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879746    4871 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:37:37.884734 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879751    4871 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:37:37.884734 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879755    4871 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:37:37.884734 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879760    4871 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:37:37.884734 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879764    4871 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:37:37.884734 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879768    4871 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:37:37.884734 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879774    4871 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:37:37.884734 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879779    4871 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:37:37.884734 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879783    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:37:37.884734 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879787    4871 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:37:37.884734 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879792    4871 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:37:37.884734 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879796    4871 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:37:37.884734 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879800    4871 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:37:37.884734 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879805    4871 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:37:37.884734 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879810    4871 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:37:37.884734 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879814    4871 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:37:37.884734 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879818    4871 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:37:37.885230 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879822    4871 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:37:37.885230 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879826    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:37:37.885230 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879830    4871 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:37:37.885230 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879835    4871 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:37:37.885230 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879839    4871 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:37:37.885230 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879844    4871 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:37:37.885230 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879850    4871 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:37:37.885230 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879857    4871 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:37:37.885230 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879862    4871 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:37:37.885230 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879866    4871 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:37:37.885230 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879871    4871 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:37:37.885230 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879876    4871 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:37:37.885230 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879881    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:37:37.885230 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879888    4871 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:37:37.885230 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879894    4871 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:37:37.885230 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879899    4871 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:37:37.885230 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879904    4871 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:37:37.885230 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879908    4871 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:37:37.885230 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879912    4871 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:37:37.885736 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879917    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:37:37.885736 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879921    4871 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:37:37.885736 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879925    4871 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:37:37.885736 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879929    4871 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:37:37.885736 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879934    4871 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:37:37.885736 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879938    4871 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:37:37.885736 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879942    4871 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:37:37.885736 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879946    4871 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:37:37.885736 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879951    4871 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:37:37.885736 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879955    4871 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:37:37.885736 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879960    4871 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:37:37.885736 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.879964    4871 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:37:37.885736 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880790    4871 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:37:37.885736 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880799    4871 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:37:37.885736 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880804    4871 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:37:37.885736 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880809    4871 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:37:37.885736 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880813    4871 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:37:37.885736 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880818    4871 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:37:37.885736 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880823    4871 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:37:37.885736 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880827    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:37:37.886290 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880832    4871 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:37:37.886290 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880836    4871 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:37:37.886290 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880841    4871 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:37:37.886290 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880845    4871 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:37:37.886290 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880850    4871 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:37:37.886290 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880855    4871 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:37:37.886290 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880860    4871 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:37:37.886290 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880864    4871 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:37:37.886290 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880868    4871 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:37:37.886290 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880873    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:37:37.886290 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880877    4871 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:37:37.886290 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880881    4871 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:37:37.886290 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880888    4871 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:37:37.886290 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880894    4871 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:37:37.886290 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880899    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:37:37.886290 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880905    4871 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:37:37.886290 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880910    4871 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:37:37.886290 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880914    4871 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:37:37.886290 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880919    4871 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:37:37.886759 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880924    4871 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:37:37.886759 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880928    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:37:37.886759 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880933    4871 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:37:37.886759 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880938    4871 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:37:37.886759 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880944    4871 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:37:37.886759 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880950    4871 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:37:37.886759 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880954    4871 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:37:37.886759 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880960    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:37:37.886759 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880964    4871 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:37:37.886759 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880969    4871 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:37:37.886759 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880975    4871 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:37:37.886759 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880979    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:37:37.886759 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880984    4871 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:37:37.886759 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880988    4871 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:37:37.886759 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880992    4871 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:37:37.886759 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.880996    4871 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:37:37.886759 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881001    4871 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:37:37.886759 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881005    4871 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:37:37.886759 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881009    4871 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:37:37.886759 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881013    4871 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:37:37.887271 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881019    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:37:37.887271 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881023    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:37:37.887271 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881027    4871 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:37:37.887271 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881031    4871 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:37:37.887271 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881036    4871 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:37:37.887271 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881040    4871 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:37:37.887271 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881044    4871 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:37:37.887271 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881048    4871 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:37:37.887271 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881053    4871 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:37:37.887271 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881057    4871 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:37:37.887271 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881061    4871 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:37:37.887271 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881065    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:37:37.887271 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881070    4871 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:37:37.887271 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881074    4871 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:37:37.887271 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881078    4871 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:37:37.887271 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881083    4871 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:37:37.887271 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881088    4871 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:37:37.887271 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881092    4871 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:37:37.887271 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881096    4871 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:37:37.887738 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881104    4871 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:37:37.887738 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881108    4871 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:37:37.887738 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881113    4871 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:37:37.887738 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881117    4871 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:37:37.887738 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881122    4871 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:37:37.887738 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881127    4871 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:37:37.887738 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881131    4871 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:37:37.887738 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881135    4871 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:37:37.887738 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881139    4871 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:37:37.887738 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881144    4871 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:37:37.887738 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881148    4871 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:37:37.887738 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881152    4871 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:37:37.887738 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881156    4871 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:37:37.887738 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881160    4871 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:37:37.887738 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881165    4871 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:37:37.887738 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881170    4871 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:37:37.887738 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881174    4871 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:37:37.887738 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881178    4871 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:37:37.887738 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881182    4871 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:37:37.887738 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881187    4871 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:37:37.888255 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881191    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:37:37.888255 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881196    4871 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:37:37.888255 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881200    4871 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:37:37.888255 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881204    4871 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:37:37.888255 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881209    4871 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:37:37.888255 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881214    4871 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:37:37.888255 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881221    4871 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:37:37.888255 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881228    4871 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:37:37.888255 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881251    4871 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:37:37.888255 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881256    4871 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:37:37.888255 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881271    4871 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:37:37.888255 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881276    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:37:37.888255 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881282    4871 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:37:37.888255 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881287    4871 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:37:37.888255 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881291    4871 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:37:37.888255 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881295    4871 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:37:37.888255 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881299    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:37:37.888255 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881303    4871 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:37:37.888255 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881307    4871 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:37:37.888724 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.881312    4871 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:37:37.888724 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881458    4871 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 21:37:37.888724 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881470    4871 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 21:37:37.888724 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881481    4871 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 21:37:37.888724 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881489    4871 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 21:37:37.888724 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881497    4871 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 21:37:37.888724 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881503    4871 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 21:37:37.888724 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881510    4871 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 21:37:37.888724 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881516    4871 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 21:37:37.888724 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881522    4871 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 21:37:37.888724 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881527    4871 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 21:37:37.888724 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881533    4871 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 21:37:37.888724 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881538    4871 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 21:37:37.888724 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881544    4871 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 21:37:37.888724 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881549    4871 flags.go:64] FLAG: --cgroup-root=""
Oct 30 21:37:37.888724 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881554    4871 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 21:37:37.888724 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881560    4871 flags.go:64] FLAG: --client-ca-file=""
Oct 30 21:37:37.888724 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881564    4871 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 21:37:37.888724 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881569    4871 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 21:37:37.888724 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881576    4871 flags.go:64] FLAG: --cluster-domain=""
Oct 30 21:37:37.888724 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881581    4871 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 21:37:37.888724 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881586    4871 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 21:37:37.888724 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881591    4871 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 21:37:37.888724 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881596    4871 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 21:37:37.889323 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881603    4871 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 21:37:37.889323 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881611    4871 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 21:37:37.889323 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881616    4871 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 21:37:37.889323 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881622    4871 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 21:37:37.889323 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881627    4871 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 21:37:37.889323 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881632    4871 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 21:37:37.889323 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881637    4871 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 21:37:37.889323 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881643    4871 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 21:37:37.889323 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881648    4871 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 21:37:37.889323 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881655    4871 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 21:37:37.889323 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881661    4871 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 21:37:37.889323 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881666    4871 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 21:37:37.889323 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881671    4871 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 21:37:37.889323 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881676    4871 flags.go:64] FLAG: --enable-server="true"
Oct 30 21:37:37.889323 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881681    4871 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 21:37:37.889323 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881689    4871 flags.go:64] FLAG: --event-burst="100"
Oct 30 21:37:37.889323 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881694    4871 flags.go:64] FLAG: --event-qps="50"
Oct 30 21:37:37.889323 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881699    4871 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 21:37:37.889323 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881704    4871 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 21:37:37.889323 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881709    4871 flags.go:64] FLAG: --eviction-hard=""
Oct 30 21:37:37.889323 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881716    4871 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 21:37:37.889323 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881720    4871 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 21:37:37.889323 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881726    4871 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 21:37:37.889323 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881731    4871 flags.go:64] FLAG: --eviction-soft=""
Oct 30 21:37:37.889323 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881736    4871 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 21:37:37.889975 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881741    4871 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 21:37:37.889975 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881746    4871 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 21:37:37.889975 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881751    4871 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 21:37:37.889975 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881756    4871 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 21:37:37.889975 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881761    4871 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 21:37:37.889975 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881765    4871 flags.go:64] FLAG: --feature-gates=""
Oct 30 21:37:37.889975 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881773    4871 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 21:37:37.889975 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881778    4871 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 21:37:37.889975 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881786    4871 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 21:37:37.889975 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881791    4871 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 21:37:37.889975 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881796    4871 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 21:37:37.889975 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881801    4871 flags.go:64] FLAG: --help="false"
Oct 30 21:37:37.889975 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881806    4871 flags.go:64] FLAG: --hostname-override="ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:37:37.889975 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881812    4871 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 21:37:37.889975 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881817    4871 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 21:37:37.889975 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881822    4871 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 21:37:37.889975 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881827    4871 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 21:37:37.889975 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881833    4871 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 21:37:37.889975 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881837    4871 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 21:37:37.889975 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881842    4871 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 21:37:37.889975 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881847    4871 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 21:37:37.889975 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881852    4871 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 21:37:37.889975 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881857    4871 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 21:37:37.890594 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881862    4871 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 21:37:37.890594 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881867    4871 flags.go:64] FLAG: --kube-reserved=""
Oct 30 21:37:37.890594 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881876    4871 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 21:37:37.890594 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881881    4871 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 21:37:37.890594 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881886    4871 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 21:37:37.890594 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881891    4871 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 21:37:37.890594 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881896    4871 flags.go:64] FLAG: --lock-file=""
Oct 30 21:37:37.890594 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881900    4871 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 21:37:37.890594 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881905    4871 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 21:37:37.890594 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881910    4871 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 21:37:37.890594 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881919    4871 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 21:37:37.890594 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881924    4871 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 21:37:37.890594 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881929    4871 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 21:37:37.890594 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881933    4871 flags.go:64] FLAG: --logging-format="text"
Oct 30 21:37:37.890594 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881938    4871 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 21:37:37.890594 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881944    4871 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 21:37:37.890594 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881949    4871 flags.go:64] FLAG: --manifest-url=""
Oct 30 21:37:37.890594 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881956    4871 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 21:37:37.890594 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881963    4871 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 21:37:37.890594 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881968    4871 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 21:37:37.890594 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881975    4871 flags.go:64] FLAG: --max-pods="110"
Oct 30 21:37:37.890594 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881981    4871 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 21:37:37.890594 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881986    4871 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 21:37:37.890594 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881990    4871 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 21:37:37.890594 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.881995    4871 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 21:37:37.891174 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882000    4871 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 21:37:37.891174 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882005    4871 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 21:37:37.891174 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882010    4871 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 21:37:37.891174 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882021    4871 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 21:37:37.891174 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882026    4871 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 21:37:37.891174 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882031    4871 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 21:37:37.891174 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882036    4871 flags.go:64] FLAG: --pod-cidr=""
Oct 30 21:37:37.891174 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882041    4871 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 21:37:37.891174 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882046    4871 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 21:37:37.891174 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882053    4871 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 21:37:37.891174 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882058    4871 flags.go:64] FLAG: --port="10250"
Oct 30 21:37:37.891174 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882063    4871 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 21:37:37.891174 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882068    4871 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-0b5ea134fd6aead4a"
Oct 30 21:37:37.891174 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882074    4871 flags.go:64] FLAG: --qos-reserved=""
Oct 30 21:37:37.891174 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882079    4871 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 21:37:37.891174 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882084    4871 flags.go:64] FLAG: --register-node="true"
Oct 30 21:37:37.891174 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882089    4871 flags.go:64] FLAG: --register-with-taints=""
Oct 30 21:37:37.891174 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882095    4871 flags.go:64] FLAG: --registry-burst="10"
Oct 30 21:37:37.891174 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882100    4871 flags.go:64] FLAG: --registry-qps="5"
Oct 30 21:37:37.891174 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882105    4871 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 21:37:37.891174 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882110    4871 flags.go:64] FLAG: --reserved-memory=""
Oct 30 21:37:37.891174 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882117    4871 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 21:37:37.891174 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882122    4871 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 21:37:37.891174 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882127    4871 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 21:37:37.891174 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882134    4871 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 21:37:37.891174 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882139    4871 flags.go:64] FLAG: --runonce="false"
Oct 30 21:37:37.891823 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882145    4871 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 21:37:37.891823 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882150    4871 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 21:37:37.891823 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882156    4871 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 21:37:37.891823 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882161    4871 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 21:37:37.891823 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882165    4871 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 21:37:37.891823 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882170    4871 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 21:37:37.891823 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882175    4871 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 21:37:37.891823 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882180    4871 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 21:37:37.891823 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882184    4871 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 21:37:37.891823 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882190    4871 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 21:37:37.891823 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882194    4871 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 21:37:37.891823 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882199    4871 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 21:37:37.891823 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882204    4871 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 21:37:37.891823 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882209    4871 flags.go:64] FLAG: --system-cgroups=""
Oct 30 21:37:37.891823 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882214    4871 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 21:37:37.891823 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882226    4871 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 21:37:37.891823 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882231    4871 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 21:37:37.891823 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882251    4871 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 21:37:37.891823 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882258    4871 flags.go:64] FLAG: --tls-min-version=""
Oct 30 21:37:37.891823 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882263    4871 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 21:37:37.891823 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882268    4871 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 21:37:37.891823 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882272    4871 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 21:37:37.891823 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882277    4871 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 21:37:37.891823 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882282    4871 flags.go:64] FLAG: --v="2"
Oct 30 21:37:37.891823 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882289    4871 flags.go:64] FLAG: --version="false"
Oct 30 21:37:37.892422 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882296    4871 flags.go:64] FLAG: --vmodule=""
Oct 30 21:37:37.892422 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882303    4871 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 21:37:37.892422 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.882308    4871 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 21:37:37.892422 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882535    4871 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:37:37.892422 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882543    4871 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:37:37.892422 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882552    4871 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:37:37.892422 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882557    4871 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:37:37.892422 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882561    4871 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:37:37.892422 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882570    4871 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:37:37.892422 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882577    4871 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:37:37.892422 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882582    4871 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:37:37.892422 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882586    4871 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:37:37.892422 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882591    4871 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:37:37.892422 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882595    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:37:37.892422 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882599    4871 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:37:37.892422 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882603    4871 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:37:37.892422 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882607    4871 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:37:37.892422 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882612    4871 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:37:37.892422 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882616    4871 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:37:37.892422 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882621    4871 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:37:37.892908 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882625    4871 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:37:37.892908 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882629    4871 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:37:37.892908 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882636    4871 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:37:37.892908 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882640    4871 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:37:37.892908 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882645    4871 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:37:37.892908 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882649    4871 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:37:37.892908 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882653    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:37:37.892908 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882657    4871 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:37:37.892908 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882662    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:37:37.892908 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882667    4871 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:37:37.892908 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882671    4871 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:37:37.892908 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882675    4871 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:37:37.892908 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882679    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:37:37.892908 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882684    4871 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:37:37.892908 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882690    4871 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:37:37.892908 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882696    4871 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:37:37.892908 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882701    4871 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:37:37.892908 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882708    4871 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:37:37.892908 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882713    4871 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:37:37.893763 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882718    4871 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:37:37.893763 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882724    4871 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:37:37.893763 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882729    4871 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:37:37.893763 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882734    4871 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:37:37.893763 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882738    4871 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:37:37.893763 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882742    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:37:37.893763 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882747    4871 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:37:37.893763 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882752    4871 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:37:37.893763 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882756    4871 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:37:37.893763 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882760    4871 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:37:37.893763 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882764    4871 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:37:37.893763 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882769    4871 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:37:37.893763 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882773    4871 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:37:37.893763 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882777    4871 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:37:37.893763 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882781    4871 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:37:37.893763 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882787    4871 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:37:37.893763 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882792    4871 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:37:37.893763 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882796    4871 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:37:37.893763 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882801    4871 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:37:37.893763 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882805    4871 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:37:37.894709 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882810    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:37:37.894709 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882814    4871 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:37:37.894709 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882818    4871 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:37:37.894709 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882823    4871 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:37:37.894709 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882827    4871 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:37:37.894709 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882831    4871 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:37:37.894709 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882835    4871 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:37:37.894709 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882839    4871 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:37:37.894709 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882844    4871 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:37:37.894709 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882848    4871 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:37:37.894709 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882854    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:37:37.894709 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882859    4871 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:37:37.894709 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882863    4871 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:37:37.894709 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882867    4871 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:37:37.894709 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882871    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:37:37.894709 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882877    4871 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:37:37.894709 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882882    4871 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:37:37.894709 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882886    4871 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:37:37.894709 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882891    4871 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:37:37.894709 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882896    4871 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:37:37.895381 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882900    4871 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:37:37.895381 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882904    4871 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:37:37.895381 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882909    4871 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:37:37.895381 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882913    4871 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:37:37.895381 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882917    4871 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:37:37.895381 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882921    4871 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:37:37.895381 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882926    4871 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:37:37.895381 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882932    4871 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:37:37.895381 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882937    4871 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:37:37.895381 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882941    4871 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:37:37.895381 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882946    4871 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:37:37.895381 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882950    4871 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:37:37.895381 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882955    4871 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:37:37.895381 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882960    4871 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:37:37.895381 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882964    4871 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:37:37.895381 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882968    4871 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:37:37.895381 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882972    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:37:37.895381 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882976    4871 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:37:37.895381 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882981    4871 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:37:37.895381 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882985    4871 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:37:37.895917 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882989    4871 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:37:37.895917 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882993    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:37:37.895917 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.882999    4871 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:37:37.895917 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883004    4871 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:37:37.895917 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883009    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:37:37.895917 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883013    4871 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:37:37.895917 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883019    4871 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:37:37.895917 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883024    4871 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:37:37.895917 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883029    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:37:37.895917 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883034    4871 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:37:37.895917 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.883080    4871 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:37:37.895917 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883139    4871 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:37:37.895917 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883147    4871 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:37:37.895917 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883152    4871 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:37:37.895917 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883156    4871 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:37:37.895917 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883161    4871 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:37:37.896368 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883165    4871 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:37:37.896368 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883170    4871 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:37:37.896368 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883175    4871 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:37:37.896368 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883179    4871 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:37:37.896368 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883187    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:37:37.896368 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883191    4871 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:37:37.896368 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883196    4871 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:37:37.896368 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883200    4871 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:37:37.896368 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883205    4871 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:37:37.896368 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883209    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:37:37.896368 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883214    4871 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:37:37.896368 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883218    4871 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:37:37.896368 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883222    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:37:37.896368 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883227    4871 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:37:37.896368 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883231    4871 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:37:37.896368 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883256    4871 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:37:37.896368 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883261    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:37:37.896368 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883265    4871 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:37:37.896368 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883272    4871 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:37:37.896368 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883277    4871 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:37:37.896902 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883281    4871 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:37:37.896902 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883285    4871 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:37:37.896902 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883289    4871 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:37:37.896902 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883296    4871 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:37:37.896902 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883302    4871 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:37:37.896902 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883309    4871 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:37:37.896902 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883314    4871 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:37:37.896902 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883319    4871 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:37:37.896902 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883324    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:37:37.896902 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883328    4871 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:37:37.896902 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883333    4871 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:37:37.896902 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883337    4871 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:37:37.896902 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883341    4871 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:37:37.896902 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883345    4871 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:37:37.896902 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883349    4871 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:37:37.896902 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883353    4871 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:37:37.896902 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883359    4871 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:37:37.896902 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883364    4871 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:37:37.896902 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883368    4871 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:37:37.897536 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883373    4871 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:37:37.897536 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883377    4871 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:37:37.897536 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883381    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:37:37.897536 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883385    4871 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:37:37.897536 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883390    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:37:37.897536 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883398    4871 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:37:37.897536 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883402    4871 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:37:37.897536 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883408    4871 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:37:37.897536 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883412    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:37:37.897536 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883416    4871 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:37:37.897536 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883423    4871 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:37:37.897536 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883430    4871 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:37:37.897536 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883435    4871 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:37:37.897536 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883440    4871 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:37:37.897536 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883444    4871 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:37:37.897536 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883449    4871 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:37:37.897536 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883454    4871 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:37:37.897536 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883458    4871 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:37:37.897536 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883462    4871 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:37:37.898025 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883468    4871 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:37:37.898025 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883473    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:37:37.898025 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883477    4871 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:37:37.898025 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883481    4871 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:37:37.898025 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883486    4871 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:37:37.898025 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883490    4871 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:37:37.898025 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883494    4871 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:37:37.898025 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883498    4871 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:37:37.898025 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883503    4871 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:37:37.898025 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883508    4871 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:37:37.898025 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883515    4871 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:37:37.898025 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883519    4871 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:37:37.898025 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883524    4871 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:37:37.898025 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883528    4871 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:37:37.898025 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883532    4871 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:37:37.898025 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883537    4871 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:37:37.898025 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883542    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:37:37.898025 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883547    4871 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:37:37.898025 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883551    4871 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:37:37.898025 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883556    4871 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:37:37.898572 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883560    4871 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:37:37.898572 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883564    4871 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:37:37.898572 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883569    4871 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:37:37.898572 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883573    4871 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:37:37.898572 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883578    4871 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:37:37.898572 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883582    4871 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:37:37.898572 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883586    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:37:37.898572 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883590    4871 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:37:37.898572 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883595    4871 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:37:37.898572 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883599    4871 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:37:37.898572 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883604    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:37:37.898572 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883608    4871 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:37:37.898572 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883612    4871 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:37:37.898572 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883617    4871 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:37:37.898572 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883622    4871 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:37:37.898572 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883627    4871 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:37:37.898572 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883631    4871 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:37:37.898572 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883635    4871 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:37:37.898572 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883641    4871 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:37:37.898572 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883646    4871 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:37:37.899043 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883650    4871 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:37:37.899043 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883655    4871 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:37:37.899043 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.883661    4871 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:37:37.899043 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.888112    4871 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 21:37:37.899043 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.888132    4871 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 21:37:37.899043 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.888149    4871 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 21:37:37.899043 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.888154    4871 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 21:37:37.899043 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888217    4871 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:37:37.899043 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888224    4871 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:37:37.899043 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888228    4871 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:37:37.899043 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888232    4871 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:37:37.899043 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888248    4871 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:37:37.899043 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888251    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:37:37.899043 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888254    4871 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:37:37.899043 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888257    4871 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:37:37.899043 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888260    4871 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:37:37.899043 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888263    4871 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:37:37.899043 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888266    4871 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:37:37.899043 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888269    4871 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:37:37.899527 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888272    4871 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:37:37.899527 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888275    4871 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:37:37.899527 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888277    4871 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:37:37.899527 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888280    4871 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:37:37.899527 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888282    4871 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:37:37.899527 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888285    4871 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:37:37.899527 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888289    4871 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:37:37.899527 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888293    4871 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:37:37.899527 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888296    4871 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:37:37.899527 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888299    4871 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:37:37.899527 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888302    4871 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:37:37.899527 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888305    4871 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:37:37.899527 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888308    4871 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:37:37.899527 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888310    4871 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:37:37.899527 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888313    4871 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:37:37.899527 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888316    4871 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:37:37.899527 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888318    4871 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:37:37.899527 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888327    4871 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:37:37.899527 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888331    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:37:37.899986 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888333    4871 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:37:37.899986 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888336    4871 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:37:37.899986 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888338    4871 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:37:37.899986 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888341    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:37:37.899986 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888344    4871 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:37:37.899986 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888346    4871 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:37:37.899986 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888349    4871 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:37:37.899986 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888351    4871 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:37:37.899986 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888354    4871 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:37:37.899986 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888356    4871 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:37:37.899986 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888359    4871 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:37:37.899986 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888362    4871 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:37:37.899986 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888364    4871 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:37:37.899986 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888367    4871 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:37:37.899986 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888370    4871 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:37:37.899986 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888372    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:37:37.899986 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888375    4871 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:37:37.899986 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888378    4871 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:37:37.899986 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888380    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:37:37.900471 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888383    4871 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:37:37.900471 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888386    4871 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:37:37.900471 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888389    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:37:37.900471 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888391    4871 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:37:37.900471 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888394    4871 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:37:37.900471 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888396    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:37:37.900471 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888399    4871 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:37:37.900471 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888402    4871 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:37:37.900471 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888404    4871 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:37:37.900471 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888407    4871 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:37:37.900471 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888410    4871 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:37:37.900471 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888413    4871 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:37:37.900471 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888421    4871 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:37:37.900471 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888425    4871 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:37:37.900471 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888427    4871 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:37:37.900471 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888430    4871 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:37:37.900471 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888432    4871 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:37:37.900471 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888435    4871 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:37:37.900471 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888437    4871 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:37:37.900931 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888440    4871 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:37:37.900931 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888442    4871 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:37:37.900931 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888445    4871 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:37:37.900931 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888447    4871 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:37:37.900931 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888450    4871 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:37:37.900931 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888452    4871 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:37:37.900931 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888455    4871 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:37:37.900931 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888458    4871 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:37:37.900931 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888460    4871 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:37:37.900931 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888463    4871 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:37:37.900931 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888466    4871 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:37:37.900931 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888469    4871 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:37:37.900931 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888471    4871 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:37:37.900931 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888474    4871 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:37:37.900931 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888477    4871 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:37:37.900931 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888480    4871 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:37:37.900931 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888482    4871 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:37:37.900931 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888485    4871 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:37:37.900931 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888487    4871 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:37:37.900931 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888489    4871 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:37:37.901437 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888492    4871 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:37:37.901437 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888494    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:37:37.901437 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888497    4871 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:37:37.901437 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888499    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:37:37.901437 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888502    4871 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:37:37.901437 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888504    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:37:37.901437 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888512    4871 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:37:37.901437 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888515    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:37:37.901437 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888518    4871 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:37:37.901437 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888520    4871 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:37:37.901437 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888523    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:37:37.901437 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888525    4871 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:37:37.901437 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888528    4871 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:37:37.901437 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888531    4871 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:37:37.901437 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888534    4871 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:37:37.901437 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888536    4871 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:37:37.901437 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888539    4871 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:37:37.901986 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.888567    4871 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:37:37.901986 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888776    4871 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:37:37.901986 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888781    4871 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:37:37.901986 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888784    4871 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:37:37.901986 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888787    4871 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:37:37.901986 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888790    4871 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:37:37.901986 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888792    4871 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:37:37.901986 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888795    4871 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:37:37.901986 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888798    4871 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:37:37.901986 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888801    4871 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:37:37.901986 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888804    4871 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:37:37.901986 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888806    4871 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:37:37.901986 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888809    4871 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:37:37.901986 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888811    4871 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:37:37.901986 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888814    4871 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:37:37.901986 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888818    4871 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:37:37.902722 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888822    4871 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:37:37.902722 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888825    4871 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:37:37.902722 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888828    4871 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:37:37.902722 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888831    4871 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:37:37.902722 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888833    4871 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:37:37.902722 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888837    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:37:37.902722 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888849    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:37:37.902722 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888852    4871 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:37:37.902722 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888855    4871 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:37:37.902722 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888857    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:37:37.902722 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888860    4871 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:37:37.902722 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888862    4871 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:37:37.902722 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888865    4871 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:37:37.902722 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888868    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:37:37.902722 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888870    4871 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:37:37.902722 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888873    4871 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:37:37.902722 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888877    4871 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:37:37.902722 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888881    4871 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:37:37.902722 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888884    4871 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:37:37.903213 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888887    4871 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:37:37.903213 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888890    4871 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:37:37.903213 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888892    4871 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:37:37.903213 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888894    4871 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:37:37.903213 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888897    4871 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:37:37.903213 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888900    4871 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:37:37.903213 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888903    4871 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:37:37.903213 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888906    4871 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:37:37.903213 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888909    4871 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:37:37.903213 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888911    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:37:37.903213 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888914    4871 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:37:37.903213 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888916    4871 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:37:37.903213 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888919    4871 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:37:37.903213 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888921    4871 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:37:37.903213 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888924    4871 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:37:37.903213 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888926    4871 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:37:37.903213 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888929    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:37:37.903213 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888931    4871 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:37:37.903213 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888934    4871 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:37:37.903213 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888936    4871 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:37:37.903707 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888944    4871 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:37:37.903707 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888947    4871 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:37:37.903707 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888950    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:37:37.903707 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888953    4871 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:37:37.903707 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888955    4871 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:37:37.903707 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888958    4871 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:37:37.903707 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888961    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:37:37.903707 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888963    4871 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:37:37.903707 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888966    4871 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:37:37.903707 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888968    4871 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:37:37.903707 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888971    4871 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:37:37.903707 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888973    4871 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:37:37.903707 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888976    4871 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:37:37.903707 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888978    4871 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:37:37.903707 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888981    4871 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:37:37.903707 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888983    4871 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:37:37.903707 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888986    4871 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:37:37.903707 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888988    4871 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:37:37.903707 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888991    4871 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:37:37.904213 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888994    4871 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:37:37.904213 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888996    4871 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:37:37.904213 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.888999    4871 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:37:37.904213 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889001    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:37:37.904213 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889004    4871 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:37:37.904213 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889006    4871 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:37:37.904213 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889009    4871 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:37:37.904213 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889011    4871 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:37:37.904213 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889014    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:37:37.904213 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889016    4871 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:37:37.904213 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889019    4871 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:37:37.904213 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889021    4871 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:37:37.904213 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889024    4871 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:37:37.904213 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889027    4871 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:37:37.904213 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889035    4871 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:37:37.904213 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889038    4871 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:37:37.904213 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889041    4871 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:37:37.904213 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889043    4871 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:37:37.904213 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889046    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:37:37.904213 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889049    4871 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:37:37.904724 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889051    4871 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:37:37.904724 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889054    4871 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:37:37.904724 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889056    4871 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:37:37.904724 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889059    4871 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:37:37.904724 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889061    4871 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:37:37.904724 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889064    4871 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:37:37.904724 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889066    4871 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:37:37.904724 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889068    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:37:37.904724 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889071    4871 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:37:37.904724 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889073    4871 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:37:37.904724 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889076    4871 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:37:37.904724 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889078    4871 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:37:37.904724 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889081    4871 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:37:37.904724 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.889103    4871 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:37:37.904724 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889153    4871 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:37:37.904724 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889157    4871 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:37:37.905131 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889160    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:37:37.905131 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889163    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:37:37.905131 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889166    4871 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:37:37.905131 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889169    4871 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:37:37.905131 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889171    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:37:37.905131 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889174    4871 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:37:37.905131 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889177    4871 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:37:37.905131 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889179    4871 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:37:37.905131 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889182    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:37:37.905131 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889184    4871 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:37:37.905131 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889187    4871 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:37:37.905131 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889198    4871 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:37:37.905131 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889202    4871 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:37:37.905131 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889206    4871 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:37:37.905131 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889209    4871 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:37:37.905131 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889212    4871 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:37:37.905131 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889214    4871 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:37:37.905131 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889217    4871 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:37:37.905131 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889220    4871 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:37:37.905622 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889223    4871 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:37:37.905622 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889225    4871 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:37:37.905622 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889228    4871 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:37:37.905622 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889231    4871 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:37:37.905622 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889250    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:37:37.905622 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889253    4871 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:37:37.905622 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889256    4871 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:37:37.905622 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889258    4871 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:37:37.905622 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889261    4871 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:37:37.905622 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889263    4871 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:37:37.905622 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889266    4871 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:37:37.905622 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889268    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:37:37.905622 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889271    4871 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:37:37.905622 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889273    4871 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:37:37.905622 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889276    4871 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:37:37.905622 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889279    4871 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:37:37.905622 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889281    4871 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:37:37.905622 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889284    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:37:37.905622 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889286    4871 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:37:37.906068 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889289    4871 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:37:37.906068 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889291    4871 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:37:37.906068 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889297    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:37:37.906068 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889299    4871 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:37:37.906068 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889302    4871 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:37:37.906068 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889305    4871 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:37:37.906068 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889313    4871 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:37:37.906068 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889316    4871 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:37:37.906068 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889319    4871 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:37:37.906068 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889321    4871 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:37:37.906068 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889324    4871 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:37:37.906068 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889327    4871 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:37:37.906068 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889329    4871 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:37:37.906068 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889332    4871 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:37:37.906068 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889335    4871 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:37:37.906068 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889337    4871 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:37:37.906068 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889341    4871 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:37:37.906068 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889344    4871 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:37:37.906068 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889346    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:37:37.906580 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889349    4871 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:37:37.906580 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889351    4871 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:37:37.906580 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889354    4871 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:37:37.906580 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889356    4871 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:37:37.906580 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889359    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:37:37.906580 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889361    4871 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:37:37.906580 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889364    4871 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:37:37.906580 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889366    4871 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:37:37.906580 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889369    4871 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:37:37.906580 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889386    4871 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:37:37.906580 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889389    4871 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:37:37.906580 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889392    4871 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:37:37.906580 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889394    4871 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:37:37.906580 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889397    4871 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:37:37.906580 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889400    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:37:37.906580 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889404    4871 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:37:37.906580 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889406    4871 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:37:37.906580 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889409    4871 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:37:37.906580 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889411    4871 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:37:37.907036 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889414    4871 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:37:37.907036 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889422    4871 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:37:37.907036 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889425    4871 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:37:37.907036 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889428    4871 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:37:37.907036 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889430    4871 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:37:37.907036 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889433    4871 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:37:37.907036 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889435    4871 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:37:37.907036 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889438    4871 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:37:37.907036 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889440    4871 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:37:37.907036 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889442    4871 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:37:37.907036 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889445    4871 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:37:37.907036 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889448    4871 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:37:37.907036 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889450    4871 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:37:37.907036 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889456    4871 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:37:37.907036 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889459    4871 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:37:37.907036 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889461    4871 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:37:37.907036 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889464    4871 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:37:37.907036 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889466    4871 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:37:37.907036 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889469    4871 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:37:37.907036 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889472    4871 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:37:37.907708 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889474    4871 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:37:37.907708 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889477    4871 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:37:37.907708 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889479    4871 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:37:37.907708 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889482    4871 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:37:37.907708 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889486    4871 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:37:37.907708 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889490    4871 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:37:37.907708 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889493    4871 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:37:37.907708 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:37.889496    4871 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:37:37.907708 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.889561    4871 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 21:37:37.907708 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.890649    4871 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 30 21:37:37.907708 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.890701    4871 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 30 21:37:37.907708 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.890905    4871 server.go:1014] "Starting client certificate rotation"
Oct 30 21:37:37.907708 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.891040    4871 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 21:37:37.907708 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.891089    4871 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 21:31:56 +0000 UTC" deadline="2028-08-09 00:25:57.907231848 +0000 UTC"
Oct 30 21:37:37.907708 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.891126    4871 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15554h48m20.016107666s"
Oct 30 21:37:37.907708 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.891434    4871 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:37:37.908442 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.892696    4871 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:37:37.908442 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.895457    4871 log.go:25] "Validated CRI v1 runtime API"
Oct 30 21:37:37.908442 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.897028    4871 log.go:25] "Validated CRI v1 image API"
Oct 30 21:37:37.908442 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.897747    4871 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 21:37:37.908442 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.899683    4871 fs.go:135] Filesystem UUIDs: map[6ac87993-cbcf-4e29-8e44-b6d395f7dc1e:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 d3d60f1a-6035-4228-8334-dbfe0aa51694:/dev/nvme0n1p4]
Oct 30 21:37:37.908693 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.899709    4871 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/047c43343c8d3a6ca7dbb44be20d27b13bc9d3a62d2c27732b32c88271a79021/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/047c43343c8d3a6ca7dbb44be20d27b13bc9d3a62d2c27732b32c88271a79021/userdata/shm major:0 minor:246 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/05c69dcfb41fe3181298c0037998566ca8c95db99104a8739c64cc106cff8214/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/05c69dcfb41fe3181298c0037998566ca8c95db99104a8739c64cc106cff8214/userdata/shm major:0 minor:240 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0c8fda6bdba774f960398c99876ee2b975ed7571ccb35945e3d16ccbe83646b9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0c8fda6bdba774f960398c99876ee2b975ed7571ccb35945e3d16ccbe83646b9/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0cc39ae0c0738db00bdba883f5a07a93a1a8a36e787b9a29a2566f92369b68da/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0cc39ae0c0738db00bdba883f5a07a93a1a8a36e787b9a29a2566f92369b68da/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/168e06a666a628bbe4b646281ae2c3b727dbb614786968762d945ffc95c1d908/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/168e06a666a628bbe4b646281ae2c3b727dbb614786968762d945ffc95c1d908/userdata/shm major:0 minor:238 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1d384fd2f42fcd3364665cf5490cfcfb2ffe9be5cc6174f2dafcb6f37d6cb55c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1d384fd2f42fcd3364665cf5490cfcfb2ffe9be5cc6174f2dafcb6f37d6cb55c/userdata/shm major:0 minor:288 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1d9a1d9c4bcaff3bf006aa1fd82d5f0ed5473e5536c00f96aa2ab48006a271eb/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1d9a1d9c4bcaff3bf006aa1fd82d5f0ed5473e5536c00f96aa2ab48006a271eb/userdata/shm major:0 minor:218 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3c5ad66e5790dfd6cd348d4f2ae9c2c4418f37047a4a9e8a5cf4de6772f91fc6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3c5ad66e5790dfd6cd348d4f2ae9c2c4418f37047a4a9e8a5cf4de6772f91fc6/userdata/shm major:0 minor:234 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3e103fa82ae87141bdaa8d02c8f8a91a33635d2868773802478d7187e4ab2200/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3e103fa82ae87141bdaa8d02c8f8a91a33635d2868773802478d7187e4ab2200/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4ebb73f893610de7c7f7c99d369364e94cee977c267a33f918b1ffed7e1d6963/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4ebb73f893610de7c7f7c99d369364e94cee977c267a33f918b1ffed7e1d6963/userdata/shm major:0 minor:215 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4f516aad48abe9950750e12d96a48abf731a56b0e54c1424bd180419fc148b9c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4f516aad48abe9950750e12d96a48abf731a56b0e54c1424bd180419fc148b9c/userdata/shm major:0 minor:290 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4fa38f1049cb90c4ec0a82e2f000d049e39133939c55cbc3f898a79fae258b78/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4fa38f1049cb90c4ec0a82e2f000d049e39133939c55cbc3f898a79fae258b78/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/722f567ea5aca5dab8d18f3865b9a6a56f2a0a9123efa1d3bd7a8efee9bc2d91/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/722f567ea5aca5dab8d18f3865b9a6a56f2a0a9123efa1d3bd7a8efee9bc2d91/userdata/shm major:0 minor:214 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8141d1060aedcae9e8276e2435ae0d51406506284af41b835416efde5b1f99fa/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8141d1060aedcae9e8276e2435ae0d51406506284af41b835416efde5b1f99fa/userdata/shm major:0 minor:242 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/92c0b7873904ff975ba9638df3112650d5becec57db5160005d933c8de8860cd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/92c0b7873904ff975ba9638df3112650d5becec57db5160005d933c8de8860cd/userdata/shm major:0 minor:243 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9d70bbf91a12f4450deb83490604353f3607520ccdbe175a3ecc4b6a93654eef/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9d70bbf91a12f4450deb83490604353f3607520ccdbe175a3ecc4b6a93654eef/userdata/shm major:0 minor:74 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c4c22dc33e25512d0c3ae645987661746b0c1ac84cf568e3250994918ee72d31/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c4c22dc33e25512d0c3ae645987661746b0c1ac84cf568e3250994918ee72d31/userdata/shm major:0 minor:70 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e1686a9277d1b295a6c32714b59c5d06fb328bad430c3643db035b87b78a6dc1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e1686a9277d1b295a6c32714b59c5d06fb328bad430c3643db035b87b78a6dc1/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e67ac7ed16cc9130b49c76aece77d8e8eec0b37d7eccde37c096a9c04de6fbe1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e67ac7ed16cc9130b49c76aece77d8e8eec0b37d7eccde37c096a9c04de6fbe1/userdata/shm major:0 minor:235 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ea38fb4da192cb36f586fb61f29453437cad03605c6d13c34528d69f83ad72cf/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ea38fb4da192cb36f586fb61f29453437cad03605c6d13c34528d69f83ad72cf/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/eec01d450a8b96b3679b58621236996f1dbf1f8666a73877528cca3ecb2abcce/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/eec01d450a8b96b3679b58621236996f1dbf1f8666a73877528cca3ecb2abcce/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f8de11d8a1076b1f125776dfbe8032416f0c5971e1bb947947ba43aa31611d31/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f8de11d8a1076b1f125776dfbe8032416f0c5971e1bb947947ba43aa31611d31/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/07b45ded-5ba8-49dc-b6ab-c170ca47af9b/volumes/kubernetes.io~projected/kube-api-access-nfcrl:{mountpoint:/var/lib/kubelet/pods/07b45ded-5ba8-49dc-b6ab-c170ca47af9b/volumes/kubernetes.io~projected/kube-api-access-nfcrl major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0dca73b7-6526-48ac-a76e-70e441debc00/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/0dca73b7-6526-48ac-a76e-70e441debc00/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:228 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0dca73b7-6526-48ac-a76e-70e441debc00/volumes/kubernetes.io~projected/kube-api-access-g8hlw:{mountpoint:/var/lib/kubelet/pods/0dca73b7-6526-48ac-a76e-70e441debc00/volumes/kubernetes.io~projected/kube-api-access-g8hlw major:0 minor:230 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0dca73b7-6526-48ac-a76e-70e441debc00/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/0dca73b7-6526-48ac-a76e-70e441debc00/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:209 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0dca73b7-6526-48ac-a76e-70e441debc00/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/0dca73b7-6526-48ac-a76e-70e441debc00/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:210 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1dac48d8-aefb-4ff4-9213-b13def641c00/volumes/kubernetes.io~projected/kube-api-access-kgdlc:{mountpoint:/var/lib/kubelet/pods/1dac48d8-aefb-4ff4-9213-b13def641c00/volumes/kubernetes.io~projected/kube-api-access-kgdlc major:0 minor:233 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2360fddc-52a5-42a5-82ca-a05aff84d217/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/2360fddc-52a5-42a5-82ca-a05aff84d217/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2360fddc-52a5-42a5-82ca-a05aff84d217/volumes/kubernetes.io~projected/kube-api-access-v7lhk:{mountpoint:/var/lib/kubelet/pods/2360fddc-52a5-42a5-82ca-a05aff84d217/volumes/kubernetes.io~projected/kube-api-access-v7lhk major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2360fddc-52a5-42a5-82ca-a05aff84d217/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/2360fddc-52a5-42a5-82ca-a05aff84d217/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:222 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e/volumes/kubernetes.io~projected/kube-api-access-d49lh:{mountpoint:/var/lib/kubelet/pods/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e/volumes/kubernetes.io~projected/kube-api-access-d49lh major:0 minor:226 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:212 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:208 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/32888534-d682-4428-90d9-03c858ce5ab8/volumes/kubernetes.io~projected/kube-api-access-zs5q4:{mountpoint:/var/lib/kubelet/pods/32888534-d682-4428-90d9-03c858ce5ab8/volumes/kubernetes.io~projected/kube-api-access-zs5q4 major:0 minor:227 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/37186e06-8d56-4c61-ba10-e0408d2e2fa4/volumes/kubernetes.io~projected/kube-api-access-zn8jz:{mountpoint:/var/lib/kubelet/pods/37186e06-8d56-4c61-ba10-e0408d2e2fa4/volumes/kubernetes.io~projected/kube-api-access-zn8jz major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/37186e06-8d56-4c61-ba10-e0408d2e2fa4/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/37186e06-8d56-4c61-ba10-e0408d2e2fa4/volumes/kubernetes.io~secret/serving-cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4086d1bd-6e47-4d50-a7f9-5be23d123d6e/volumes/kubernetes.io~projected/kube-api-access-qmxkp:{mountpoint:/var/lib/kubelet/pods/4086d1bd-6e47-4d50-a7f9-5be23d123d6e/volumes/kubernetes.io~projected/kube-api-access-qmxkp major:0 minor:224 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4086d1bd-6e47-4d50-a7f9-5be23d123d6e/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/4086d1bd-6e47-4d50-a7f9-5be23d123d6e/volumes/kubernetes.io~secret/serving-cert major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/457fd859-0880-43f8-b712-2fd90c9debb0/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/457fd859-0880-43f8-b712-2fd90c9debb0/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/457fd859-0880-43f8-b712-2fd90c9debb0/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/457fd859-0880-43f8-b712-2fd90c9debb0/volumes/kubernetes.io~empty-dir/tmp major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/457fd859-0880-43f8-b712-2fd90c9debb0/volumes/kubernetes.io~projected/kube-api-access-6lnqk:{mountpoint:/var/lib/kubelet/pods/457fd859-0880-43f8-b712-2fd90c9debb0/volumes/kubernetes.io~projected/kube-api-access-6lnqk major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/51e670da-3db6-412b-8753-8606d7a20fe3/volumes/kubernetes.io~projected/kube-api-access-mlggr:{mountpoint:/var/lib/kubelet/pods/51e670da-3db6-412b-8753-8606d7a20fe3/volumes/kubernetes.io~projected/kube-api-access-mlggr major:0 minor:223 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee/volumes/kubernetes.io~projected/kube-api-access-2zgrn:{mountpoint:/var/lib/kubelet/pods/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee/volumes/kubernetes.io~projected/kube-api-access-2zgrn major:0 minor:321 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5cf90653-8f33-4bb2-b622-1dae9144a9fe/volumes/kubernetes.io~projected/kube-api-access-xbs5d:{mountpoint:/var/lib/kubelet/pods/5cf90653-8f33-4bb2-b622-1dae9144a9fe/volumes/kubernetes.io~projected/kube-api-access-xbs5d major:0 minor:220 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5f5757a0-0a11-4637-9aa5-4e283792017d/volumes/kubernetes.io~projected/kube-api-access-phfjb:{mountpoint:/var/lib/kubelet/pods/5f5757a0-0a11-4637-9aa5-4e283792017d/volumes/kubernetes.io~projected/kube-api-access-phfjb major:0 minor:205 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/65827025-dc76-42d8-be9f-bdfed6b28429/volumes/kubernetes.io~projected/kube-api-access-kh4k2:{mountpoint:/var/lib/kubelet/pods/65827025-dc76-42d8-be9f-bdfed6b28429/volumes/kubernetes.io~projected/kube-api-access-kh4k2 major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6d9d43c2-40da-47a8-adb4-d3ba378d77d9/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/6d9d43c2-40da-47a8-adb4-d3ba378d77d9/volumes/kubernetes.io~secret/agent-certs major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6e615be3-41d7-4f82-90d7-7ce71ed9190b/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/6e615be3-41d7-4f82-90d7-7ce71ed9190b/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:282 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/726700d7-c252-479e-abdd-18c0573102d1/volumes/kubernetes.io~projected/kube-api-access-z4xm6:{mountpoint:/var/lib/kubelet/pods/726700d7-c252-479e-abdd-18c0573102d1/volumes/kubernetes.io~projected/kube-api-access-z4xm6 major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2/volumes/kubernetes.io~projected/kube-api-access-jxsh7:{mountpoint:/var/lib/kubelet/pods/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2/volumes/kubernetes.io~projected/kube-api-access-jxsh7 major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2/volumes/kubernetes.io~projected/kube-api-access-574vn:{mountpoint:/var/lib/kubelet/pods/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2/volumes/kubernetes.io~projected/kube-api-access-574vn major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/872bccf3-b52e-4c52-9f5e-0487b4957b50/volumes/kubernetes.io~projected/kube-api-access-5crvk:{mountpoint:/var/lib/kubelet/pods/872bccf3-b52e-4c52-9f5e-0487b4957b50/volumes/kubernetes.io~projected/kube-api-access-5crvk major:0 minor:231 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/909906d5-5049-4742-9b5a-b9c03ed5cc39/volumes/kubernetes.io~projected/kube-api-access-vrvv4:{mountpoint:/var/lib/kubelet/pods/909906d5-5049-4742-9b5a-b9c03ed5cc39/volumes/kubernetes.io~projected/kube-api-access-vrvv4 major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/948affe7-aaf8-4d3e-a515-ed9552d19d90/volumes/kubernetes.io~projected/kube-api-access-wv5hx:{mountpoint:/var/lib/kubelet/pods/948affe7-aaf8-4d3e-a515-ed9552d19d90/volumes/kubernetes.io~projected/kube-api-access-wv5hx major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9eefa84c-5304-4c2a-89cf-3304097ecf42/volumes/kubernetes.io~projected/kube-api-access-gxmv7:{mountpoint:/var/lib/kubelet/pods/9eefa84c-5304-4c2a-89cf-3304097ecf42/volumes/kubernetes.io~projected/kube-api-access-gxmv7 major:0 minor:287 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b7002d6d-834c-4bdc-907e-805edd577a40/volumes/kubernetes.io~projected/kube-api-access-c4qb5:{mountpoint:/var/lib/kubelet/pods/b7002d6d-834c-4bdc-907e-805edd577a40/volumes/kubernetes.io~projected/kube-api-access-c4qb5 major:0 minor:221 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b7002d6d-834c-4bdc-907e-805edd577a40/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/b7002d6d-834c-4bdc-907e-805edd577a40/volumes/kubernetes.io~secret/serving-cert major:0 minor:213 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/be4fa9bf-41e0-4b26-9457-f9815a36d5b5/volumes/kubernetes.io~projected/kube-api-access-vpmml:{mountpoint:/var/lib/kubelet/pods/be4fa9bf-41e0-4b26-9457-f9815a36d5b5/volumes/kubernetes.io~projected/kube-api-access-vpmml major:0 minor:229 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/be4fa9bf-41e0-4b26-9457-f9815a36d5b5/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/be4fa9bf-41e0-4b26-9457-f9815a36d5b5/volumes/kubernetes.io~secret/cert major:0 minor:211 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cb8bde01-2bc6-4301-aa2f-2858158bd54e/volumes/kubernetes.io~projected/kube-api-access-qmlvz:{mountpoint:/var/lib/kubelet/pods/cb8bde01-2bc6-4301-aa2f-2858158bd54e/volumes/kubernetes.io~projected/kube-api-access-qmlvz major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d54af961-c593-4707-9cec-1069000bce50/volumes/kubernetes.io~projected/kube-api-access-7kbb8:{mountpoint:/var/lib/kubelet/pods/d54af961-c593-4707-9cec-1069000bce50/volumes/kubernetes.io~projected/kube-api-access-7kbb8 major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e0535562-96d7-4fc4-a5d5-ad4cc785080e/volumes/kubernetes.io~projected/kube-api-access-f26sz:{mountpoint:/var/lib/kubelet/pods/e0535562-96d7-4fc4-a5d5-ad4cc785080e/volumes/kubernetes.io~projected/kube-api-access-f26sz major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e0535562-96d7-4fc4-a5d5-ad4cc785080e/volumes/kubernetes.io~secret/default-certificate:{mountpoint:/var/lib/kubelet/pods/e0535562-96d7-4fc4-a5d5-ad4cc785080e/volumes/kubernetes.io~secret/default-certificate major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e0535562-96d7-4fc4-a5d5-ad4cc785080e/volumes/kubernetes.io~secret/stats-auth:{mountpoint:/var/lib/kubelet/pods/e0535562-96d7-4fc4-a5d5-ad4cc785080e/volumes/kubernetes.io~secret/stats-auth major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fb70ad86-4b45-4a53-a3cb-1f6b59640708/volumes/kubernetes.io~projected/kube-api-access-7976h:{mountpoint:/var/lib/kubelet/pods/fb70ad86-4b45-4a53-a3cb-1f6b59640708/volumes/kubernetes.io~projected/kube-api-access-7976h major:0 minor:232 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fb70ad86-4b45-4a53-a3cb-1f6b59640708/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/fb70ad86-4b45-4a53-a3cb-1f6b59640708/volumes/kubernetes.io~secret/serving-cert major:0 minor:207 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/40b3b8c61b65c8fb8c3076dddcc436722a41cb6a6b1f6755d2d4c59458ddea33/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/5a0be31dc627b8a016ab110ca281df7e175395fbe746b1407b26b919bdc54192/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/282275d6e5bdade1eed4791eb06c780d18b8016b2d3a7ee9d91e6b79e7d7af42/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-109:{mountpoint:/var/lib/containers/storage/overlay/0662df8262a9cef6d8856669be654978adb028646b26ff25cb04b1ea7862bd1a/merged major:0 minor:109 fsType:overlay blockSize:0} overlay_0-111:{mountpoint:/var/lib/containers/storage/overlay/c19594ad0c72db08665e42f8f2b016c50ee1b08bd3b2b0784687ae18453d5fac/merged major:0 minor:111 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/ae0610402cb5fd06a54d45636b30f1efb5aa5cb6ed0723771c4542368383fd2f/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/f80569236faed0a21e74ecd3caf8e126a71d4af6774b3e9e8ba753a610126035/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/31e0644f98357b031e1c67a349c085f8f4108044f7e72c7df6c4c5cef04c19ee/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/00b3356226ac02786749bde9d62cb90cb734997e2d430922149adcb0f9a0560c/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/f74fcbfc338bc78a42c30df15f27fe20d403e5f2261bf8b699a76574c9572d33/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-144:{mountpoint:/var/lib/containers/storage/overlay/0226e5235ba8fd7a0d748a1e5204b30b149dbc7a8ee5a622854e9c177d0e883b/merged major:0 minor:144 fsType:overlay blockSize:0} overlay_0-146:{mountpoint:/var/lib/containers/storage/overlay/b75da909fc6b0ebb8c4463837aeb5e7d0df669da49cec5ba5f5bab5ea9374e05/merged major:0 minor:146 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/dfe9b496c9194cf211320aac52b6ac71d1be01672aa1425518d5f22778be0963/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/eb7b0d90e45b6f37386ace54c95f928478bcc2a9c436e27119c24114a02f9524/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/48bebd21bd2472d5751b1a721938a86783265dce5c3f8a862250c6a270b30786/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/d3d45b7ecfd41eb08c7df16d53a23b82cdbd60086767db4c53d050cf47c37c18/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/7bda511d42f1d85ee822b848dbf7eb6a07445c1eacdf05ee0d0c4d63b83ed1fc/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/74e7d06bdffb4a3a63f08bdc37aa1ea7b6220b8582fff245c7bf6ecedd227f7f/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/bd86e40f649119dbe939a268fa9e2d65e658108053718792b51b9f538f3db10b/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/362ed400aa378078dab5d3a584c9ebec21664b7add87bb935c4fd8464a983828/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/38c63137f4ca8051162c6dd91dc6f3fd33ee123fde8272275aaa17ec49b8b167/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/1d37a244304b0fe2acda2660db7fa78803c4935bde8771be3c6c97657f61a4b3/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-250:{mountpoint:/var/lib/containers/storage/overlay/bab174854284005690ae34e350560d1137834ea5c704acfb5471c7b14538f12e/merged major:0 minor:250 fsType:overlay blockSize:0} overlay_0-252:{mountpoint:/var/lib/containers/storage/overlay/dc8c46a588bcad259048b47d89142f93e6d75b06ffb85883477964955f3bdc91/merged major:0 minor:252 fsType:overlay blockSize:0} overlay_0-254:{mountpoint:/var/lib/containers/storage/overlay/043a3bc5569e2d9eb0f10e07851e3f31412489d830036f63e2495d5d7e7a45df/merged major:0 minor:254 fsType:overlay blockSize:0} overlay_0-256:{mountpoint:/var/lib/containers/storage/overlay/f1708ae1ec706d3eab697f2ba71190dd37c5c4cc332ac0f00a46e0e239cb8e1f/merged major:0 minor:256 fsType:overlay blockSize:0} overlay_0-258:{mountpoint:/var/lib/containers/storage/overlay/3471e7873ebad05575d7c0615e566c1bb825b4b921bd8aff5fb5d79a26a4e155/merged major:0 minor:258 fsType:overlay blockSize:0} overlay_0-260:{mountpoint:/var/lib/containers/storage/overlay/d84e9e7065fe49c860109e924ee8a9d602f0a368c41f1347c71031ef1919f089/merged major:0 minor:260 fsType:overlay blockSize:0} overlay_0-262:{mountpoint:/var/lib/containers/storage/overlay/18a79a002b8d1a59b612492ee885fb0ab75f479e3fb94b7621b9db6c3027d4ad/merged major:0 minor:262 fsType:overlay blockSize:0} overlay_0-264:{mountpoint:/var/lib/containers/storage/overlay/eb9894117ebd908cc9132cbb40b29ed825c843aff2176299106bb42c41114e26/merged major:0 minor:264 fsType:overlay blockSize:0} overlay_0-274:{mountpoint:/var/lib/containers/storage/overlay/d32c7dfbba9f07113f06213970a5ad7e3ac97999e4107a9d52db2b8f1a83d085/merged major:0 minor:274 fsType:overlay blockSize:0} overlay_0-276:{mountpoint:/var/lib/containers/storage/overlay/1a2e5f10dcb119ab43e7c0e5c0b585dad08490a5255a71d320e4e3491102f086/merged major:0 minor:276 fsType:overlay blockSize:0} overlay_0-278:{mountpoint:/var/lib/containers/storage/overlay/2b3bd72cd41058ecf2a4bb36fd1b1e49e4385f8db69ede03b418fa104a247039/merged major:0 minor:278 fsType:overlay blockSize:0} overlay_0-280:{mountpoint:/var/lib/containers/storage/overlay/ba315a5bcb29c89a062805e52c9998427d88564fcb97e009141099b0a182260f/merged major:0 minor:280 fsType:overlay blockSize:0} overlay_0-292:{mountpoint:/var/lib/containers/storage/overlay/9a18541ef9e382afe9fe2b1a8916745b2fcb8136318437adc386ab15ed1583b4/merged major:0 minor:292 fsType:overlay blockSize:0} overlay_0-294:{mountpoint:/var/lib/containers/storage/overlay/00ee3a4411f665485bd006b1f2e8faf95e8ca2c49219732121a7f12afeea24d4/merged major:0 minor:294 fsType:overlay blockSize:0} overlay_0-296:{mountpoint:/var/lib/containers/storage/overlay/35035d5e8182c09a47a8a95782188512be75c0db27b3052c14e8d6251d030ae0/merged major:0 minor:296 fsType:overlay blockSize:0} overlay_0-298:{mountpoint:/var/lib/containers/storage/overlay/ec935bb90ec391123b236bc189bc2e50ee018804371059cf165a79ae97cae0cd/merged major:0 minor:298 fsType:overlay blockSize:0} overlay_0-304:{mountpoint:/var/lib/containers/storage/overlay/d79a5915b70d8bfd9d90d83f564739f70102a6b08487efb79c38a4206f798b8e/merged major:0 minor:304 fsType:overlay blockSize:0} overlay_0-310:{mountpoint:/var/lib/containers/storage/overlay/84c5953f1a411919d36c60755767d7362638e13108d6e127b66c66092ea787cc/merged major:0 minor:310 fsType:overlay blockSize:0} overlay_0-312:{mountpoint:/var/lib/containers/storage/overlay/cc878c72da89f2a2336e0c28f37b9de6e747ac10de0de1915923a9a3cf78ed3c/merged major:0 minor:312 fsType:overlay blockSize:0} overlay_0-314:{mountpoint:/var/lib/containers/storage/overlay/fce507560709df2d48ad9721ff512314996e2e07b952a0aa9e39b892a5ebf133/merged major:0 minor:314 fsType:overlay blockSize:0} overlay_0-329:{mountpoint:/var/lib/containers/storage/overlay/3c1b5f111f5b336fb5f3871d017128bbdbad61fe2df6949a4c83a2ea0a6fcc5d/merged major:0 minor:329 fsType:overlay blockSize:0} overlay_0-335:{mountpoint:/var/lib/containers/storage/overlay/b0f4f801b19e88584333d9e5273fec084147cb8cf488be7099aafdc15eabf80c/merged major:0 minor:335 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/f9b3496c61f98ca4b9058864d4cb0a5bf0882c66d9d7be43f152c031e3efa661/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/21a3c41f52513ea5f47f6efbc870834b2897a365f9682e75df50c4f73c00e5e6/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-68:{mountpoint:/var/lib/containers/storage/overlay/bf0beec6e3fb54b99b4e7ba216293cf4c9acc84e813d3e5a59e5769467b1fd61/merged major:0 minor:68 fsType:overlay blockSize:0} overlay_0-72:{mountpoint:/var/lib/containers/storage/overlay/771b44f211ba02a97046c6a93776bebd0d253660ded07d846c4a925da70ca632/merged major:0 minor:72 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/02d891b0903a4e831717c307e331a7976f4f71b5406b8c560ebca99496c058cd/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/3e13f587c9f14b9ab073ad9f0130252d8b05ebfd333192bf7def96fe6dcf6dac/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/bec1a8509870838445eaafa0660ee9e9b4dda396aeec51c3d28ad4426576c31c/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/3e35fba965620f92418625ece77dc82e329189bc5eb0ecd7d38a7c083ebfda70/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/7814665143ea5d1177f9a5ab9c948170e3340e054a45cdbe0483153316af118a/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/8435c76835b4c73e260e879a7d85001c1f323406748f5bc2db8f3a28df51b4a5/merged major:0 minor:96 fsType:overlay blockSize:0}]
Oct 30 21:37:37.908996 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.907069    4871 manager.go:217] Machine: {Timestamp:2026-10-30 21:37:37.906609121 +0000 UTC m=+0.084097729 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3088732 MemoryCapacity:33164341248 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a6fbaec3b62344ab46bc006672533 SystemUUID:ec2a6fba-ec3b-6234-4ab4-6bc006672533 BootID:661fa28e-b79d-4a20-b743-5bee4b6c0176 Filesystems:[{Device:/var/lib/kubelet/pods/e0535562-96d7-4fc4-a5d5-ad4cc785080e/volumes/kubernetes.io~secret/stats-auth DeviceMajor:0 DeviceMinor:197 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/be4fa9bf-41e0-4b26-9457-f9815a36d5b5/volumes/kubernetes.io~projected/kube-api-access-vpmml DeviceMajor:0 DeviceMinor:229 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2360fddc-52a5-42a5-82ca-a05aff84d217/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:57 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:212 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9eefa84c-5304-4c2a-89cf-3304097ecf42/volumes/kubernetes.io~projected/kube-api-access-gxmv7 DeviceMajor:0 DeviceMinor:287 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e1686a9277d1b295a6c32714b59c5d06fb328bad430c3643db035b87b78a6dc1/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/be4fa9bf-41e0-4b26-9457-f9815a36d5b5/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:211 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/b7002d6d-834c-4bdc-907e-805edd577a40/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:213 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/5cf90653-8f33-4bb2-b622-1dae9144a9fe/volumes/kubernetes.io~projected/kube-api-access-xbs5d DeviceMajor:0 DeviceMinor:220 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fb70ad86-4b45-4a53-a3cb-1f6b59640708/volumes/kubernetes.io~projected/kube-api-access-7976h DeviceMajor:0 DeviceMinor:232 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8141d1060aedcae9e8276e2435ae0d51406506284af41b835416efde5b1f99fa/userdata/shm DeviceMajor:0 DeviceMinor:242 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-294 DeviceMajor:0 DeviceMinor:294 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:208 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1dac48d8-aefb-4ff4-9213-b13def641c00/volumes/kubernetes.io~projected/kube-api-access-kgdlc DeviceMajor:0 DeviceMinor:233 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-262 DeviceMajor:0 DeviceMinor:262 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-292 DeviceMajor:0 DeviceMinor:292 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-256 DeviceMajor:0 DeviceMinor:256 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2360fddc-52a5-42a5-82ca-a05aff84d217/volumes/kubernetes.io~projected/kube-api-access-v7lhk DeviceMajor:0 DeviceMinor:66 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fb70ad86-4b45-4a53-a3cb-1f6b59640708/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:207 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0dca73b7-6526-48ac-a76e-70e441debc00/volumes/kubernetes.io~projected/kube-api-access-g8hlw DeviceMajor:0 DeviceMinor:230 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-314 DeviceMajor:0 DeviceMinor:314 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:222 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/92c0b7873904ff975ba9638df3112650d5becec57db5160005d933c8de8860cd/userdata/shm DeviceMajor:0 DeviceMinor:243 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c4c22dc33e25512d0c3ae645987661746b0c1ac84cf568e3250994918ee72d31/userdata/shm DeviceMajor:0 DeviceMinor:70 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/047c43343c8d3a6ca7dbb44be20d27b13bc9d3a62d2c27732b32c88271a79021/userdata/shm DeviceMajor:0 DeviceMinor:246 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-280 DeviceMajor:0 DeviceMinor:280 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-298 DeviceMajor:0 DeviceMinor:298 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582172672 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/726700d7-c252-479e-abdd-18c0573102d1/volumes/kubernetes.io~projected/kube-api-access-z4xm6 DeviceMajor:0 DeviceMinor:61 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9d70bbf91a12f4450deb83490604353f3607520ccdbe175a3ecc4b6a93654eef/userdata/shm DeviceMajor:0 DeviceMinor:74 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4fa38f1049cb90c4ec0a82e2f000d049e39133939c55cbc3f898a79fae258b78/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/eec01d450a8b96b3679b58621236996f1dbf1f8666a73877528cca3ecb2abcce/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/457fd859-0880-43f8-b712-2fd90c9debb0/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:56 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/6d9d43c2-40da-47a8-adb4-d3ba378d77d9/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:58 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/5f5757a0-0a11-4637-9aa5-4e283792017d/volumes/kubernetes.io~projected/kube-api-access-phfjb DeviceMajor:0 DeviceMinor:205 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-276 DeviceMajor:0 DeviceMinor:276 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-329 DeviceMajor:0 DeviceMinor:329 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-335 DeviceMajor:0 DeviceMinor:335 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/457fd859-0880-43f8-b712-2fd90c9debb0/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:55 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0dca73b7-6526-48ac-a76e-70e441debc00/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:228 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-264 DeviceMajor:0 DeviceMinor:264 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f8de11d8a1076b1f125776dfbe8032416f0c5971e1bb947947ba43aa31611d31/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2/volumes/kubernetes.io~projected/kube-api-access-574vn DeviceMajor:0 DeviceMinor:67 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-111 DeviceMajor:0 DeviceMinor:111 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b7002d6d-834c-4bdc-907e-805edd577a40/volumes/kubernetes.io~projected/kube-api-access-c4qb5 DeviceMajor:0 DeviceMinor:221 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/872bccf3-b52e-4c52-9f5e-0487b4957b50/volumes/kubernetes.io~projected/kube-api-access-5crvk DeviceMajor:0 DeviceMinor:231 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4086d1bd-6e47-4d50-a7f9-5be23d123d6e/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:206 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0dca73b7-6526-48ac-a76e-70e441debc00/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:210 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/457fd859-0880-43f8-b712-2fd90c9debb0/volumes/kubernetes.io~projected/kube-api-access-6lnqk DeviceMajor:0 DeviceMinor:60 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e/volumes/kubernetes.io~projected/kube-api-access-d49lh DeviceMajor:0 DeviceMinor:226 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0cc39ae0c0738db00bdba883f5a07a93a1a8a36e787b9a29a2566f92369b68da/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-72 DeviceMajor:0 DeviceMinor:72 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0c8fda6bdba774f960398c99876ee2b975ed7571ccb35945e3d16ccbe83646b9/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/51e670da-3db6-412b-8753-8606d7a20fe3/volumes/kubernetes.io~projected/kube-api-access-mlggr DeviceMajor:0 DeviceMinor:223 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-258 DeviceMajor:0 DeviceMinor:258 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-296 DeviceMajor:0 DeviceMinor:296 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-310 DeviceMajor:0 DeviceMinor:310 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/948affe7-aaf8-4d3e-a515-ed9552d19d90/volumes/kubernetes.io~projected/kube-api-access-wv5hx DeviceMajor:0 DeviceMinor:63 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ea38fb4da192cb36f586fb61f29453437cad03605c6d13c34528d69f83ad72cf/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/2360fddc-52a5-42a5-82ca-a05aff84d217/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-254 DeviceMajor:0 DeviceMinor:254 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/909906d5-5049-4742-9b5a-b9c03ed5cc39/volumes/kubernetes.io~projected/kube-api-access-vrvv4 DeviceMajor:0 DeviceMinor:59 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-146 DeviceMajor:0 DeviceMinor:146 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/65827025-dc76-42d8-be9f-bdfed6b28429/volumes/kubernetes.io~projected/kube-api-access-kh4k2 DeviceMajor:0 DeviceMinor:225 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-252 DeviceMajor:0 DeviceMinor:252 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d54af961-c593-4707-9cec-1069000bce50/volumes/kubernetes.io~projected/kube-api-access-7kbb8 DeviceMajor:0 DeviceMinor:204 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e0535562-96d7-4fc4-a5d5-ad4cc785080e/volumes/kubernetes.io~projected/kube-api-access-f26sz DeviceMajor:0 DeviceMinor:202 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-274 DeviceMajor:0 DeviceMinor:274 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cb8bde01-2bc6-4301-aa2f-2858158bd54e/volumes/kubernetes.io~projected/kube-api-access-qmlvz DeviceMajor:0 DeviceMinor:62 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-250 DeviceMajor:0 DeviceMinor:250 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-260 DeviceMajor:0 DeviceMinor:260 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3e103fa82ae87141bdaa8d02c8f8a91a33635d2868773802478d7187e4ab2200/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-278 DeviceMajor:0 DeviceMinor:278 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6e615be3-41d7-4f82-90d7-7ce71ed9190b/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:282 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e0535562-96d7-4fc4-a5d5-ad4cc785080e/volumes/kubernetes.io~secret/default-certificate DeviceMajor:0 DeviceMinor:201 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1d9a1d9c4bcaff3bf006aa1fd82d5f0ed5473e5536c00f96aa2ab48006a271eb/userdata/shm DeviceMajor:0 DeviceMinor:218 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/32888534-d682-4428-90d9-03c858ce5ab8/volumes/kubernetes.io~projected/kube-api-access-zs5q4 DeviceMajor:0 DeviceMinor:227 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e67ac7ed16cc9130b49c76aece77d8e8eec0b37d7eccde37c096a9c04de6fbe1/userdata/shm DeviceMajor:0 DeviceMinor:235 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/168e06a666a628bbe4b646281ae2c3b727dbb614786968762d945ffc95c1d908/userdata/shm DeviceMajor:0 DeviceMinor:238 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-312 DeviceMajor:0 DeviceMinor:312 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-109 DeviceMajor:0 DeviceMinor:109 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/37186e06-8d56-4c61-ba10-e0408d2e2fa4/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:45 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0dca73b7-6526-48ac-a76e-70e441debc00/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:209 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/4086d1bd-6e47-4d50-a7f9-5be23d123d6e/volumes/kubernetes.io~projected/kube-api-access-qmxkp DeviceMajor:0 DeviceMinor:224 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1d384fd2f42fcd3364665cf5490cfcfb2ffe9be5cc6174f2dafcb6f37d6cb55c/userdata/shm DeviceMajor:0 DeviceMinor:288 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/37186e06-8d56-4c61-ba10-e0408d2e2fa4/volumes/kubernetes.io~projected/kube-api-access-zn8jz DeviceMajor:0 DeviceMinor:203 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/722f567ea5aca5dab8d18f3865b9a6a56f2a0a9123efa1d3bd7a8efee9bc2d91/userdata/shm DeviceMajor:0 DeviceMinor:214 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3c5ad66e5790dfd6cd348d4f2ae9c2c4418f37047a4a9e8a5cf4de6772f91fc6/userdata/shm DeviceMajor:0 DeviceMinor:234 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-304 DeviceMajor:0 DeviceMinor:304 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4ebb73f893610de7c7f7c99d369364e94cee977c267a33f918b1ffed7e1d6963/userdata/shm DeviceMajor:0 DeviceMinor:215 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/05c69dcfb41fe3181298c0037998566ca8c95db99104a8739c64cc106cff8214/userdata/shm DeviceMajor:0 DeviceMinor:240 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4f516aad48abe9950750e12d96a48abf731a56b0e54c1424bd180419fc148b9c/userdata/shm DeviceMajor:0 DeviceMinor:290 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/07b45ded-5ba8-49dc-b6ab-c170ca47af9b/volumes/kubernetes.io~projected/kube-api-access-nfcrl DeviceMajor:0 DeviceMinor:64 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-68 DeviceMajor:0 DeviceMinor:68 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-144 DeviceMajor:0 DeviceMinor:144 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee/volumes/kubernetes.io~projected/kube-api-access-2zgrn DeviceMajor:0 DeviceMinor:321 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2/volumes/kubernetes.io~projected/kube-api-access-jxsh7 DeviceMajor:0 DeviceMinor:65 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:047c43343c8d3a6 MacAddress:f6:bf:43:9f:12:9f Speed:10000 Mtu:8901} {Name:05c69dcfb41fe31 MacAddress:4a:2a:8d:bf:c7:e2 Speed:10000 Mtu:8901} {Name:168e06a666a628b MacAddress:26:ba:9e:20:bf:53 Speed:10000 Mtu:8901} {Name:1d384fd2f42fcd3 MacAddress:86:9c:96:ad:c4:85 Speed:10000 Mtu:8901} {Name:1d9a1d9c4bcaff3 MacAddress:4a:21:51:a4:6f:e8 Speed:10000 Mtu:8901} {Name:3c5ad66e5790dfd MacAddress:06:57:9f:97:a4:33 Speed:10000 Mtu:8901} {Name:4ebb73f893610de MacAddress:26:02:23:68:ff:4c Speed:10000 Mtu:8901} {Name:4f516aad48abe99 MacAddress:9a:00:3d:05:4e:03 Speed:10000 Mtu:8901} {Name:722f567ea5aca5d MacAddress:92:27:79:f4:97:94 Speed:10000 Mtu:8901} {Name:8141d1060aedcae MacAddress:3e:82:4a:bd:5d:3d Speed:10000 Mtu:8901} {Name:92c0b7873904ff9 MacAddress:de:48:89:55:00:39 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:06:ff:fc:54:30:7f Speed:0 Mtu:9001} {Name:br-int MacAddress:ce:38:e3:0a:11:48 Speed:0 Mtu:8901} {Name:e67ac7ed16cc913 MacAddress:d6:d5:84:ee:82:f6 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:06:ff:fc:54:30:7f Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:3a:84:e8:bb:ef:39 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:85:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:6a:15:04:48:34:95 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164341248 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 21:37:37.908996 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.907418    4871 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 21:37:37.908996 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.907463    4871 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 21:37:37.908996 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.907811    4871 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 21:37:37.908996 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.907836    4871 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-46.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 21:37:37.908996 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.908038    4871 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 21:37:37.908996 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.908051    4871 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 21:37:37.908996 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.908070    4871 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:37:37.908996 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.908090    4871 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:37:37.908996 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.908117    4871 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 21:37:37.908996 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.908382    4871 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 21:37:37.908996 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.908533    4871 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 21:37:37.908996 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.908581    4871 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 21:37:37.908996 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.908597    4871 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 21:37:37.908996 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.908612    4871 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 21:37:37.908996 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.908625    4871 kubelet.go:405] "Adding apiserver pod source"
Oct 30 21:37:37.908996 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.908638    4871 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 21:37:37.909991 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.909643    4871 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 21:37:37.909991 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.909669    4871 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:37:37.910229 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.910200    4871 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 21:37:37.913898 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.913880    4871 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 21:37:37.913898 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.913894    4871 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:37:37.914009 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.913995    4871 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 30 21:37:37.914193 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.914181    4871 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 21:37:37.914230 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.914212    4871 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 21:37:37.914328 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.914318    4871 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 21:37:37.914370 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.914333    4871 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 21:37:37.914370 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.914340    4871 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 21:37:37.914370 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.914345    4871 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 21:37:37.914370 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.914351    4871 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 21:37:37.914370 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.914358    4871 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 21:37:37.914370 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.914364    4871 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 21:37:37.914370 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.914369    4871 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 21:37:37.914632 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.914376    4871 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 21:37:37.914632 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.914382    4871 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 21:37:37.914632 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.914391    4871 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 21:37:37.914632 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.914400    4871 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 21:37:37.914632 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.914424    4871 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 21:37:37.914632 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.914429    4871 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 21:37:37.917834 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.917821    4871 server.go:1263] "Started kubelet"
Oct 30 21:37:37.918276 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.917905    4871 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 21:37:37.918276 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.917916    4871 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 21:37:37.918276 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.918003    4871 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 21:37:37.918458 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.918353    4871 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 21:37:37.919186 ip-10-0-139-46 systemd[1]: Started Kubernetes Kubelet.
Oct 30 21:37:37.919335 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.919181    4871 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 21:37:37.925263 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.921843    4871 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 21:37:37.925263 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.921959    4871 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 21:37:37.925263 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.921971    4871 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 21:37:37.925263 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.922064    4871 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 21:37:37.925263 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:37.922476    4871 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-46.us-west-2.compute.internal\" not found"
Oct 30 21:37:37.925263 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.923549    4871 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 21:37:37.925263 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.923622    4871 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 21:31:58 +0000 UTC" deadline="2028-08-09 22:30:39.039166994 +0000 UTC"
Oct 30 21:37:37.925263 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.923663    4871 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15576h53m1.11550794s"
Oct 30 21:37:37.925737 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.925597    4871 factory.go:55] Registering systemd factory
Oct 30 21:37:37.925737 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.925619    4871 factory.go:223] Registration of the systemd container factory successfully
Oct 30 21:37:37.927702 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.927666    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="65827025-dc76-42d8-be9f-bdfed6b28429" volumeName="kubernetes.io/configmap/65827025-dc76-42d8-be9f-bdfed6b28429-telemetry-config" seLinuxMountContext=""
Oct 30 21:37:37.927818 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.927714    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="948affe7-aaf8-4d3e-a515-ed9552d19d90" volumeName="kubernetes.io/projected/948affe7-aaf8-4d3e-a515-ed9552d19d90-kube-api-access-wv5hx" seLinuxMountContext=""
Oct 30 21:37:37.927818 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.927726    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cb8bde01-2bc6-4301-aa2f-2858158bd54e" volumeName="kubernetes.io/configmap/cb8bde01-2bc6-4301-aa2f-2858158bd54e-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 30 21:37:37.927818 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.927738    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7b99dea5-32ac-4c1c-af68-b5d2b7af23c2" volumeName="kubernetes.io/projected/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-kube-api-access-574vn" seLinuxMountContext=""
Oct 30 21:37:37.927818 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.927755    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cb8bde01-2bc6-4301-aa2f-2858158bd54e" volumeName="kubernetes.io/projected/cb8bde01-2bc6-4301-aa2f-2858158bd54e-kube-api-access-qmlvz" seLinuxMountContext=""
Oct 30 21:37:37.927818 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.927764    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="51e670da-3db6-412b-8753-8606d7a20fe3" volumeName="kubernetes.io/projected/51e670da-3db6-412b-8753-8606d7a20fe3-kube-api-access-mlggr" seLinuxMountContext=""
Oct 30 21:37:37.927818 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.927786    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="565e807a-a9c3-4f1b-8de0-9ff4d713a1ee" volumeName="kubernetes.io/empty-dir/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-data-volume" seLinuxMountContext=""
Oct 30 21:37:37.927818 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.927814    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b7002d6d-834c-4bdc-907e-805edd577a40" volumeName="kubernetes.io/configmap/b7002d6d-834c-4bdc-907e-805edd577a40-config" seLinuxMountContext=""
Oct 30 21:37:37.928358 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.927828    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb70ad86-4b45-4a53-a3cb-1f6b59640708" volumeName="kubernetes.io/configmap/fb70ad86-4b45-4a53-a3cb-1f6b59640708-config" seLinuxMountContext=""
Oct 30 21:37:37.928358 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.927837    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0dca73b7-6526-48ac-a76e-70e441debc00" volumeName="kubernetes.io/secret/0dca73b7-6526-48ac-a76e-70e441debc00-installation-pull-secrets" seLinuxMountContext=""
Oct 30 21:37:37.928358 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.927857    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2360fddc-52a5-42a5-82ca-a05aff84d217" volumeName="kubernetes.io/secret/2360fddc-52a5-42a5-82ca-a05aff84d217-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 30 21:37:37.928358 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.927865    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30931e26-479e-4ddb-8ce3-9b2b52dc0e8e" volumeName="kubernetes.io/configmap/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-trusted-ca" seLinuxMountContext=""
Oct 30 21:37:37.928358 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.927872    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30931e26-479e-4ddb-8ce3-9b2b52dc0e8e" volumeName="kubernetes.io/empty-dir/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-ca-trust-extracted" seLinuxMountContext=""
Oct 30 21:37:37.928358 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.927894    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5cf90653-8f33-4bb2-b622-1dae9144a9fe" volumeName="kubernetes.io/projected/5cf90653-8f33-4bb2-b622-1dae9144a9fe-kube-api-access-xbs5d" seLinuxMountContext=""
Oct 30 21:37:37.928358 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.927904    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6d9d43c2-40da-47a8-adb4-d3ba378d77d9" volumeName="kubernetes.io/secret/6d9d43c2-40da-47a8-adb4-d3ba378d77d9-agent-certs" seLinuxMountContext=""
Oct 30 21:37:37.928358 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.927931    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0dca73b7-6526-48ac-a76e-70e441debc00" volumeName="kubernetes.io/secret/0dca73b7-6526-48ac-a76e-70e441debc00-image-registry-private-configuration" seLinuxMountContext=""
Oct 30 21:37:37.928358 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.927963    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="37186e06-8d56-4c61-ba10-e0408d2e2fa4" volumeName="kubernetes.io/projected/37186e06-8d56-4c61-ba10-e0408d2e2fa4-kube-api-access-zn8jz" seLinuxMountContext=""
Oct 30 21:37:37.928358 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.927985    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6d9d43c2-40da-47a8-adb4-d3ba378d77d9" volumeName="kubernetes.io/configmap/6d9d43c2-40da-47a8-adb4-d3ba378d77d9-konnectivity-ca" seLinuxMountContext=""
Oct 30 21:37:37.928358 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.927992    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b7002d6d-834c-4bdc-907e-805edd577a40" volumeName="kubernetes.io/secret/b7002d6d-834c-4bdc-907e-805edd577a40-serving-cert" seLinuxMountContext=""
Oct 30 21:37:37.928358 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928041    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="be4fa9bf-41e0-4b26-9457-f9815a36d5b5" volumeName="kubernetes.io/secret/be4fa9bf-41e0-4b26-9457-f9815a36d5b5-cert" seLinuxMountContext=""
Oct 30 21:37:37.928358 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928120    4871 factory.go:153] Registering CRI-O factory
Oct 30 21:37:37.928358 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928157    4871 factory.go:223] Registration of the crio container factory successfully
Oct 30 21:37:37.928358 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928169    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2" volumeName="kubernetes.io/projected/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-kube-api-access-jxsh7" seLinuxMountContext=""
Oct 30 21:37:37.928358 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928218    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0dca73b7-6526-48ac-a76e-70e441debc00" volumeName="kubernetes.io/empty-dir/0dca73b7-6526-48ac-a76e-70e441debc00-ca-trust-extracted" seLinuxMountContext=""
Oct 30 21:37:37.928358 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928223    4871 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 21:37:37.928358 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928281    4871 factory.go:103] Registering Raw factory
Oct 30 21:37:37.928358 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928286    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2360fddc-52a5-42a5-82ca-a05aff84d217" volumeName="kubernetes.io/configmap/2360fddc-52a5-42a5-82ca-a05aff84d217-env-overrides" seLinuxMountContext=""
Oct 30 21:37:37.928358 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928298    4871 manager.go:1199] Started watching for new ooms in manager
Oct 30 21:37:37.928358 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928312    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2360fddc-52a5-42a5-82ca-a05aff84d217" volumeName="kubernetes.io/configmap/2360fddc-52a5-42a5-82ca-a05aff84d217-ovnkube-script-lib" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928380    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5f5757a0-0a11-4637-9aa5-4e283792017d" volumeName="kubernetes.io/projected/5f5757a0-0a11-4637-9aa5-4e283792017d-kube-api-access-phfjb" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928423    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="948affe7-aaf8-4d3e-a515-ed9552d19d90" volumeName="kubernetes.io/configmap/948affe7-aaf8-4d3e-a515-ed9552d19d90-serviceca" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928438    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb70ad86-4b45-4a53-a3cb-1f6b59640708" volumeName="kubernetes.io/secret/fb70ad86-4b45-4a53-a3cb-1f6b59640708-serving-cert" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928509    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2360fddc-52a5-42a5-82ca-a05aff84d217" volumeName="kubernetes.io/projected/2360fddc-52a5-42a5-82ca-a05aff84d217-kube-api-access-v7lhk" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928550    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="32888534-d682-4428-90d9-03c858ce5ab8" volumeName="kubernetes.io/projected/32888534-d682-4428-90d9-03c858ce5ab8-kube-api-access-zs5q4" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928566    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="726700d7-c252-479e-abdd-18c0573102d1" volumeName="kubernetes.io/configmap/726700d7-c252-479e-abdd-18c0573102d1-cni-binary-copy" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928591    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="726700d7-c252-479e-abdd-18c0573102d1" volumeName="kubernetes.io/projected/726700d7-c252-479e-abdd-18c0573102d1-kube-api-access-z4xm6" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928609    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="872bccf3-b52e-4c52-9f5e-0487b4957b50" volumeName="kubernetes.io/configmap/872bccf3-b52e-4c52-9f5e-0487b4957b50-config-volume" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928620    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="909906d5-5049-4742-9b5a-b9c03ed5cc39" volumeName="kubernetes.io/projected/909906d5-5049-4742-9b5a-b9c03ed5cc39-kube-api-access-vrvv4" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928647    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4086d1bd-6e47-4d50-a7f9-5be23d123d6e" volumeName="kubernetes.io/secret/4086d1bd-6e47-4d50-a7f9-5be23d123d6e-serving-cert" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928660    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="872bccf3-b52e-4c52-9f5e-0487b4957b50" volumeName="kubernetes.io/empty-dir/872bccf3-b52e-4c52-9f5e-0487b4957b50-tmp-dir" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928684    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="07b45ded-5ba8-49dc-b6ab-c170ca47af9b" volumeName="kubernetes.io/projected/07b45ded-5ba8-49dc-b6ab-c170ca47af9b-kube-api-access-nfcrl" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928712    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30931e26-479e-4ddb-8ce3-9b2b52dc0e8e" volumeName="kubernetes.io/configmap/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-certificates" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928725    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="37186e06-8d56-4c61-ba10-e0408d2e2fa4" volumeName="kubernetes.io/secret/37186e06-8d56-4c61-ba10-e0408d2e2fa4-serving-cert" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928761    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a099b798-61cd-4306-85ba-85c7deb25a18" volumeName="kubernetes.io/configmap/a099b798-61cd-4306-85ba-85c7deb25a18-nginx-conf" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928770    4871 manager.go:319] Starting recovery of all containers
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928772    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e0535562-96d7-4fc4-a5d5-ad4cc785080e" volumeName="kubernetes.io/secret/e0535562-96d7-4fc4-a5d5-ad4cc785080e-stats-auth" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928784    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="65827025-dc76-42d8-be9f-bdfed6b28429" volumeName="kubernetes.io/projected/65827025-dc76-42d8-be9f-bdfed6b28429-kube-api-access-kh4k2" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928801    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4086d1bd-6e47-4d50-a7f9-5be23d123d6e" volumeName="kubernetes.io/configmap/4086d1bd-6e47-4d50-a7f9-5be23d123d6e-service-ca-bundle" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928814    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="457fd859-0880-43f8-b712-2fd90c9debb0" volumeName="kubernetes.io/empty-dir/457fd859-0880-43f8-b712-2fd90c9debb0-etc-tuned" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928831    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb70ad86-4b45-4a53-a3cb-1f6b59640708" volumeName="kubernetes.io/projected/fb70ad86-4b45-4a53-a3cb-1f6b59640708-kube-api-access-7976h" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928846    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4086d1bd-6e47-4d50-a7f9-5be23d123d6e" volumeName="kubernetes.io/projected/4086d1bd-6e47-4d50-a7f9-5be23d123d6e-kube-api-access-qmxkp" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928861    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="872bccf3-b52e-4c52-9f5e-0487b4957b50" volumeName="kubernetes.io/projected/872bccf3-b52e-4c52-9f5e-0487b4957b50-kube-api-access-5crvk" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928873    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9eefa84c-5304-4c2a-89cf-3304097ecf42" volumeName="kubernetes.io/projected/9eefa84c-5304-4c2a-89cf-3304097ecf42-kube-api-access-gxmv7" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928888    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b7002d6d-834c-4bdc-907e-805edd577a40" volumeName="kubernetes.io/projected/b7002d6d-834c-4bdc-907e-805edd577a40-kube-api-access-c4qb5" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928907    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="be4fa9bf-41e0-4b26-9457-f9815a36d5b5" volumeName="kubernetes.io/projected/be4fa9bf-41e0-4b26-9457-f9815a36d5b5-kube-api-access-vpmml" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928929    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cb8bde01-2bc6-4301-aa2f-2858158bd54e" volumeName="kubernetes.io/configmap/cb8bde01-2bc6-4301-aa2f-2858158bd54e-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928946    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d54af961-c593-4707-9cec-1069000bce50" volumeName="kubernetes.io/projected/d54af961-c593-4707-9cec-1069000bce50-kube-api-access-7kbb8" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928959    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4086d1bd-6e47-4d50-a7f9-5be23d123d6e" volumeName="kubernetes.io/configmap/4086d1bd-6e47-4d50-a7f9-5be23d123d6e-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928971    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4086d1bd-6e47-4d50-a7f9-5be23d123d6e" volumeName="kubernetes.io/empty-dir/4086d1bd-6e47-4d50-a7f9-5be23d123d6e-snapshots" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.928988    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="07b45ded-5ba8-49dc-b6ab-c170ca47af9b" volumeName="kubernetes.io/configmap/07b45ded-5ba8-49dc-b6ab-c170ca47af9b-iptables-alerter-script" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.929007    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0dca73b7-6526-48ac-a76e-70e441debc00" volumeName="kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-bound-sa-token" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.929027    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="457fd859-0880-43f8-b712-2fd90c9debb0" volumeName="kubernetes.io/empty-dir/457fd859-0880-43f8-b712-2fd90c9debb0-tmp" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.929086    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5cf90653-8f33-4bb2-b622-1dae9144a9fe" volumeName="kubernetes.io/configmap/5cf90653-8f33-4bb2-b622-1dae9144a9fe-samples-operator-config" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.929100    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b7002d6d-834c-4bdc-907e-805edd577a40" volumeName="kubernetes.io/configmap/b7002d6d-834c-4bdc-907e-805edd577a40-trusted-ca" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.929117    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="909906d5-5049-4742-9b5a-b9c03ed5cc39" volumeName="kubernetes.io/empty-dir/909906d5-5049-4742-9b5a-b9c03ed5cc39-tmp-dir" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.929129    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0dca73b7-6526-48ac-a76e-70e441debc00" volumeName="kubernetes.io/configmap/0dca73b7-6526-48ac-a76e-70e441debc00-registry-certificates" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.929144    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2360fddc-52a5-42a5-82ca-a05aff84d217" volumeName="kubernetes.io/configmap/2360fddc-52a5-42a5-82ca-a05aff84d217-ovnkube-config" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.929160    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30931e26-479e-4ddb-8ce3-9b2b52dc0e8e" volumeName="kubernetes.io/secret/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-installation-pull-secrets" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.929172    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30931e26-479e-4ddb-8ce3-9b2b52dc0e8e" volumeName="kubernetes.io/secret/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-image-registry-private-configuration" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.929190    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="726700d7-c252-479e-abdd-18c0573102d1" volumeName="kubernetes.io/configmap/726700d7-c252-479e-abdd-18c0573102d1-multus-daemon-config" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.929203    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0dca73b7-6526-48ac-a76e-70e441debc00" volumeName="kubernetes.io/configmap/0dca73b7-6526-48ac-a76e-70e441debc00-trusted-ca" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.929215    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0dca73b7-6526-48ac-a76e-70e441debc00" volumeName="kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-kube-api-access-g8hlw" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.929233    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1dac48d8-aefb-4ff4-9213-b13def641c00" volumeName="kubernetes.io/projected/1dac48d8-aefb-4ff4-9213-b13def641c00-kube-api-access-kgdlc" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.929289    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="37186e06-8d56-4c61-ba10-e0408d2e2fa4" volumeName="kubernetes.io/configmap/37186e06-8d56-4c61-ba10-e0408d2e2fa4-config" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.929309    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="565e807a-a9c3-4f1b-8de0-9ff4d713a1ee" volumeName="kubernetes.io/projected/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-kube-api-access-2zgrn" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.929323    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cb8bde01-2bc6-4301-aa2f-2858158bd54e" volumeName="kubernetes.io/configmap/cb8bde01-2bc6-4301-aa2f-2858158bd54e-cni-binary-copy" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.929336    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e0535562-96d7-4fc4-a5d5-ad4cc785080e" volumeName="kubernetes.io/projected/e0535562-96d7-4fc4-a5d5-ad4cc785080e-kube-api-access-f26sz" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.929368    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e0535562-96d7-4fc4-a5d5-ad4cc785080e" volumeName="kubernetes.io/secret/e0535562-96d7-4fc4-a5d5-ad4cc785080e-default-certificate" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.929399    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6e615be3-41d7-4f82-90d7-7ce71ed9190b" volumeName="kubernetes.io/secret/6e615be3-41d7-4f82-90d7-7ce71ed9190b-original-pull-secret" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.929425    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30931e26-479e-4ddb-8ce3-9b2b52dc0e8e" volumeName="kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-bound-sa-token" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.929447    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30931e26-479e-4ddb-8ce3-9b2b52dc0e8e" volumeName="kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-kube-api-access-d49lh" seLinuxMountContext=""
Oct 30 21:37:37.929441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.929462    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4086d1bd-6e47-4d50-a7f9-5be23d123d6e" volumeName="kubernetes.io/empty-dir/4086d1bd-6e47-4d50-a7f9-5be23d123d6e-tmp" seLinuxMountContext=""
Oct 30 21:37:37.931988 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.929483    4871 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="457fd859-0880-43f8-b712-2fd90c9debb0" volumeName="kubernetes.io/projected/457fd859-0880-43f8-b712-2fd90c9debb0-kube-api-access-6lnqk" seLinuxMountContext=""
Oct 30 21:37:37.931988 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.929497    4871 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 21:37:37.931988 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.929505    4871 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 21:37:37.935544 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.935517    4871 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 21:37:37.943949 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.943922    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/1.log"
Oct 30 21:37:37.944570 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.944549    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/1.log"
Oct 30 21:37:37.952043 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.952019    4871 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 21:37:37.952043 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.952044    4871 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 21:37:37.952180 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.952065    4871 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 21:37:37.952180 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:37.952109    4871 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 30 21:37:37.952653 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.952634    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzntd_2360fddc-52a5-42a5-82ca-a05aff84d217/ovn-acl-logging/0.log"
Oct 30 21:37:37.954377 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.954354    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzntd_2360fddc-52a5-42a5-82ca-a05aff84d217/ovn-acl-logging/0.log"
Oct 30 21:37:37.962844 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.962817    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzntd_2360fddc-52a5-42a5-82ca-a05aff84d217/ovn-acl-logging/0.log"
Oct 30 21:37:37.963175 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.963146    4871 generic.go:356] "Generic (PLEG): container finished" podID="2360fddc-52a5-42a5-82ca-a05aff84d217" containerID="327a6e103e9b22f0e47ff17afc2a9a0f00bfd68b28b5cf9ce0dac0a2449d99c2" exitCode=1
Oct 30 21:37:37.969535 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.969513    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/1.log"
Oct 30 21:37:37.969657 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.969539    4871 generic.go:356] "Generic (PLEG): container finished" podID="b7002d6d-834c-4bdc-907e-805edd577a40" containerID="b357894139be464ca4c0abc3e8e4b114e23bf2f0cc6df8213aaa1e84a3a6e451" exitCode=255
Oct 30 21:37:37.982037 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.981996    4871 generic.go:356] "Generic (PLEG): container finished" podID="cb8bde01-2bc6-4301-aa2f-2858158bd54e" containerID="acecb3853fc6c693b108236e9b840bd7464447cf4201b998563860b55c1f275c" exitCode=0
Oct 30 21:37:37.982037 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.982032    4871 generic.go:356] "Generic (PLEG): container finished" podID="cb8bde01-2bc6-4301-aa2f-2858158bd54e" containerID="aac1cec8a74591c97a22b7944c1b5192d7ba619ffe3dbcb3fe6f44171b0b5ed6" exitCode=0
Oct 30 21:37:37.982037 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.982042    4871 generic.go:356] "Generic (PLEG): container finished" podID="cb8bde01-2bc6-4301-aa2f-2858158bd54e" containerID="e2ba3ba6ed4014ad938f8b91b8c7d0dc641be61bc040b2ee1d40e0eb6dd965e2" exitCode=0
Oct 30 21:37:37.982315 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.982051    4871 generic.go:356] "Generic (PLEG): container finished" podID="cb8bde01-2bc6-4301-aa2f-2858158bd54e" containerID="3eef80e415e775a8594f1d1354606ac0b273be2df4d70aa16ba324292d2a1a67" exitCode=0
Oct 30 21:37:37.982315 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.982060    4871 generic.go:356] "Generic (PLEG): container finished" podID="cb8bde01-2bc6-4301-aa2f-2858158bd54e" containerID="bb52b4c2ed8745a45397007c68e36cfeb009c46a29227600601612fb4d14dc1c" exitCode=0
Oct 30 21:37:37.982315 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.982069    4871 generic.go:356] "Generic (PLEG): container finished" podID="cb8bde01-2bc6-4301-aa2f-2858158bd54e" containerID="66659b05ba33b40eb5c5627af97ed2d6430bbcc4aa97e469eae5e44d2774c2e7" exitCode=0
Oct 30 21:37:37.985386 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:37.985349    4871 generic.go:356] "Generic (PLEG): container finished" podID="85a4f26b44393e519ecc5f1c6d0f8225" containerID="26ced90e504e27dc1a49d45e3c1b57df3ccfef918fd6108829246632096ba5d0" exitCode=0
Oct 30 21:37:38.016906 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.016883    4871 manager.go:324] Recovery completed
Oct 30 21:37:38.022599 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:38.022577    4871 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-46.us-west-2.compute.internal\" not found"
Oct 30 21:37:38.034634 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.034611    4871 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:37:38.035944 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.035921    4871 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:37:38.036019 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.035958    4871 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:37:38.036019 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.035994    4871 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:37:38.039020 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.039002    4871 cpu_manager.go:226] "Starting" policy="none"
Oct 30 21:37:38.039020 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.039014    4871 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 21:37:38.039151 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.039026    4871 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:37:38.039151 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.039032    4871 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 21:37:38.039205 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.039170    4871 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 30 21:37:38.039205 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.039180    4871 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 30 21:37:38.039205 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.039202    4871 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 30 21:37:38.039322 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.039210    4871 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 30 21:37:38.039322 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.039215    4871 policy_none.go:50] "Start"
Oct 30 21:37:38.039322 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.039224    4871 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 21:37:38.039322 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.039248    4871 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 21:37:38.039429 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.039402    4871 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 30 21:37:38.039429 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.039413    4871 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 30 21:37:38.039429 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.039421    4871 policy_none.go:44] "Start"
Oct 30 21:37:38.039989 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.039972    4871 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:37:38.040051 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.039999    4871 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:37:38.040051 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.040009    4871 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:37:38.043868 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.043841    4871 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 21:37:38.046518 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.046498    4871 manager.go:341] "Starting Device Plugin manager"
Oct 30 21:37:38.046607 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:38.046530    4871 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 21:37:38.046607 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.046545    4871 server.go:87] "Starting device plugin registration server"
Oct 30 21:37:38.046813 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.046800    4871 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 21:37:38.046869 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.046813    4871 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 21:37:38.046916 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.046888    4871 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 21:37:38.047009 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.046978    4871 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 21:37:38.047075 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.047019    4871 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 21:37:38.047075 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.047026    4871 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 21:37:38.047172 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.047081    4871 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T21:37:38.047008124Z","UUID":"93003e26-b20d-4cac-8ce8-b87ed4c9c8ac","Handler":null,"Name":"","Endpoint":""}
Oct 30 21:37:38.048361 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.048343    4871 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 21:37:38.048467 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.048366    4871 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 21:37:38.048659 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:38.048637    4871 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 21:37:38.048762 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:38.048674    4871 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-46.us-west-2.compute.internal\" not found"
Oct 30 21:37:38.056985 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.052196    4871 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-46.us-west-2.compute.internal"]
Oct 30 21:37:38.056985 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.052815    4871 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:37:38.056985 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.053677    4871 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:37:38.056985 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.053698    4871 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:37:38.056985 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.053716    4871 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:37:38.056985 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.054015    4871 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:37:38.056985 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.054151    4871 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:37:38.056985 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.054703    4871 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:37:38.056985 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.054730    4871 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:37:38.056985 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.054853    4871 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:37:38.056985 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.055029    4871 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:37:38.056985 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.055049    4871 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:37:38.056985 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.055068    4871 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:37:38.056985 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:38.055217    4871 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-46.us-west-2.compute.internal\" not found" node="ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:37:38.056985 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.055833    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-46.us-west-2.compute.internal" event={"ID":"1d836aa5280c51c365d2d598465f9d5c","Type":"ContainerStarted","Data":"a3489bcd07eaac036a281d6878c4e8be7fab01b45fee9df3bbb4943394fb9627"}
Oct 30 21:37:38.056985 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.055892    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-46.us-west-2.compute.internal" event={"ID":"1d836aa5280c51c365d2d598465f9d5c","Type":"ContainerStarted","Data":"3e103fa82ae87141bdaa8d02c8f8a91a33635d2868773802478d7187e4ab2200"}
Oct 30 21:37:38.056985 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.055918    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal" event={"ID":"85a4f26b44393e519ecc5f1c6d0f8225","Type":"ContainerStarted","Data":"8ba5c0a58e456e5c683dab20b394e024d17ef3f4d3b3f0c0c216fc9d55ada8c3"}
Oct 30 21:37:38.056985 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.055939    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal" event={"ID":"85a4f26b44393e519ecc5f1c6d0f8225","Type":"ContainerDied","Data":"26ced90e504e27dc1a49d45e3c1b57df3ccfef918fd6108829246632096ba5d0"}
Oct 30 21:37:38.056985 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.055965    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal" event={"ID":"85a4f26b44393e519ecc5f1c6d0f8225","Type":"ContainerStarted","Data":"f8de11d8a1076b1f125776dfbe8032416f0c5971e1bb947947ba43aa31611d31"}
Oct 30 21:37:38.056985 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.056126    4871 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:37:38.057768 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.057536    4871 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:37:38.057768 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.057566    4871 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:37:38.057768 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.057579    4871 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:37:38.057768 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:38.057723    4871 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-46.us-west-2.compute.internal\" not found" node="ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:37:38.067810 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.067789    4871 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:37:38.115958 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.115918    4871 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 30 21:37:38.115958 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.115930    4871 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:37:38.123002 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.122984    4871 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:37:38.147299 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.147168    4871 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:37:38.148398 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.148378    4871 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:37:38.148516 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.148415    4871 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:37:38.148516 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.148427    4871 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:37:38.148516 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.148471    4871 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:37:38.163015 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.162985    4871 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:37:38.163135 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.163055    4871 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:37:38.213039 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.213011    4871 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:37:38.213211 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:38.213058    4871 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:37:38.213211 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.213071    4871 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:37:38.216117 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.216093    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/85a4f26b44393e519ecc5f1c6d0f8225-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal\" (UID: \"85a4f26b44393e519ecc5f1c6d0f8225\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:37:38.216270 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.216120    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1d836aa5280c51c365d2d598465f9d5c-config\") pod \"kube-apiserver-proxy-ip-10-0-139-46.us-west-2.compute.internal\" (UID: \"1d836aa5280c51c365d2d598465f9d5c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:37:38.216270 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.216139    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/85a4f26b44393e519ecc5f1c6d0f8225-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal\" (UID: \"85a4f26b44393e519ecc5f1c6d0f8225\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:37:38.261754 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.261723    4871 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:37:38.261897 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.261797    4871 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:37:38.262399 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.262381    4871 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeNotReady"
Oct 30 21:37:38.262448 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.262415    4871 setters.go:546] "Node became not ready" node="ip-10-0-139-46.us-west-2.compute.internal" condition={"type":"Ready","status":"False","lastHeartbeatTime":"2026-10-30T21:37:38Z","lastTransitionTime":"2026-10-30T21:37:38Z","reason":"KubeletNotReady","message":"CSINode is not yet initialized"}
Oct 30 21:37:38.277899 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.277873    4871 nodeinfomanager.go:471] existing CSINode "ip-10-0-139-46.us-west-2.compute.internal" is owned by different node (oldNodeID="7fd386c3-cc2f-49c9-b9d3-7b9e0be72a99", newNodeID=""), cleaning up...
Oct 30 21:37:38.296149 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.296122    4871 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:37:38.296289 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:38.296175    4871 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-139-46.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:37:38.316921 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.316895    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/85a4f26b44393e519ecc5f1c6d0f8225-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal\" (UID: \"85a4f26b44393e519ecc5f1c6d0f8225\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:37:38.317001 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.316930    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/85a4f26b44393e519ecc5f1c6d0f8225-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal\" (UID: \"85a4f26b44393e519ecc5f1c6d0f8225\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:37:38.317001 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.316951    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1d836aa5280c51c365d2d598465f9d5c-config\") pod \"kube-apiserver-proxy-ip-10-0-139-46.us-west-2.compute.internal\" (UID: \"1d836aa5280c51c365d2d598465f9d5c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:37:38.317079 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.317004    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1d836aa5280c51c365d2d598465f9d5c-config\") pod \"kube-apiserver-proxy-ip-10-0-139-46.us-west-2.compute.internal\" (UID: \"1d836aa5280c51c365d2d598465f9d5c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:37:38.317079 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.317006    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/85a4f26b44393e519ecc5f1c6d0f8225-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal\" (UID: \"85a4f26b44393e519ecc5f1c6d0f8225\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:37:38.317079 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.317010    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/85a4f26b44393e519ecc5f1c6d0f8225-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal\" (UID: \"85a4f26b44393e519ecc5f1c6d0f8225\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:37:38.909797 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:38.909755    4871 apiserver.go:52] "Watching apiserver"
Oct 30 21:37:39.005203 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.005173    4871 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:37:39.054334 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.054312    4871 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 21:37:39.055009 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.054980    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-139-46.us-west-2.compute.internal","openshift-multus/multus-ggqpf","kube-system/kas-connection-checker-685895c7c9-nvwdl","openshift-image-registry/image-registry-7d487d746-hf6pr","openshift-network-operator/iptables-alerter-2wb5p","openshift-ingress-canary/ingress-canary-rqxfc","openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-7lnsg","openshift-network-console/networking-console-plugin-86c478bb88-5lpq9","openshift-network-diagnostics/network-check-source-6f78d59f99-58v4l","kube-system/kas-connection-checker-685895c7c9-tlfjh","openshift-cluster-node-tuning-operator/tuned-f9gtx","openshift-insights/insights-runtime-extractor-h6hzg","openshift-ovn-kubernetes/ovnkube-node-wzntd","openshift-network-diagnostics/network-check-target-wpc6f","openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q2pns","openshift-service-ca-operator/service-ca-operator-544d749c8c-qqxdq","openshift-image-registry/node-ca-grk5c","openshift-console-operator/console-operator-7cd56b8dbc-7tf9f","kube-system/global-pull-secret-syncer-z9h2r","openshift-dns/node-resolver-dbpfd","openshift-insights/insights-operator-7c4986d449-pf9px","openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-jgspb","kube-system/konnectivity-agent-4vb42","openshift-multus/multus-additional-cni-plugins-4frz9","openshift-multus/network-metrics-daemon-l9kxl","openshift-monitoring/cluster-monitoring-operator-884576c57-wltlg","openshift-dns/dns-default-6q6pm","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9","openshift-image-registry/image-registry-55fbc4d8d7-w4mrq","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal","openshift-ingress/router-default-57f669cfc9-d755h","kube-system/kas-connection-checker-685895c7c9-px9dx"]
Oct 30 21:37:39.056735 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.056713    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q2pns"
Oct 30 21:37:39.056968 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.056936    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wltlg"
Oct 30 21:37:39.058324 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.058302    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:39.058641 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.058627    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:39.060094 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.060071    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-5lpq9"
Oct 30 21:37:39.062598 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.062571    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:39.063599 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.063583    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9kxl"
Oct 30 21:37:39.067948 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.067928    4871 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:37:39.068022 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.067979    4871 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-46.us-west-2.compute.internal"
Oct 30 21:37:39.068818 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.068799    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6q6pm"
Oct 30 21:37:39.069705 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.069689    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:39.106877 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.106848    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Oct 30 21:37:39.107114 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.107097    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Oct 30 21:37:39.109882 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.109862    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Oct 30 21:37:39.124841 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.124818    4871 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 21:37:39.208852 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.208759    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Oct 30 21:37:39.208852 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.208785    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Oct 30 21:37:39.208852 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.208817    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-x8xlv\""
Oct 30 21:37:39.208852 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.208821    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 21:37:39.208852 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.208850    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 21:37:39.209164 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.208907    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-t84mq\""
Oct 30 21:37:39.209164 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.208970    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 21:37:39.209164 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.208978    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Oct 30 21:37:39.209164 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.209001    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:37:39.209441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.209423    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Oct 30 21:37:39.209483 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.209446    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Oct 30 21:37:39.209483 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.209457    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Oct 30 21:37:39.209483 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.209458    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 21:37:39.209579 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.209483    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 21:37:39.209674 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.209651    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Oct 30 21:37:39.209674 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.209654    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 21:37:39.209844 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.209683    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 21:37:39.209844 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.209685    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 21:37:39.209844 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.209703    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Oct 30 21:37:39.209844 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.209736    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 21:37:39.209844 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.209745    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Oct 30 21:37:39.209844 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.209763    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-cds9h\""
Oct 30 21:37:39.209844 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.209773    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 21:37:39.209844 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.209765    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Oct 30 21:37:39.209844 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.209672    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:37:39.210183 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.210162    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Oct 30 21:37:39.210263 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.210252    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:37:39.210387 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.210358    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 21:37:39.210956 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.210941    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 21:37:39.211008 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.210949    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 21:37:39.211182 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.211168    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 21:37:39.211231 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.211175    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 21:37:39.211342 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.211330    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 21:37:39.211423 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.211412    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 21:37:39.211513 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.211500    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 21:37:39.211645 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.211630    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 21:37:39.211700 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.211653    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 21:37:39.211746 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.211727    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 21:37:39.211802 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.211748    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:37:39.211802 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.211764    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 21:37:39.211802 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.211749    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Oct 30 21:37:39.211802 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.211749    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Oct 30 21:37:39.211983 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.211779    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Oct 30 21:37:39.211983 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.211826    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Oct 30 21:37:39.211983 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.211832    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Oct 30 21:37:39.211983 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.211850    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Oct 30 21:37:39.211983 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.211855    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:37:39.211983 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.211859    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 21:37:39.211983 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.211897    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Oct 30 21:37:39.211983 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.211906    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 30 21:37:39.211983 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.211925    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Oct 30 21:37:39.211983 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.211932    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:37:39.211983 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.211935    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Oct 30 21:37:39.211983 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.211960    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 21:37:39.212388 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.211992    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 21:37:39.212388 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.212034    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 21:37:39.212388 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.212055    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Oct 30 21:37:39.212388 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.212069    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 21:37:39.212388 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.212073    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 21:37:39.212388 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.212142    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 21:37:39.212388 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.212169    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 21:37:39.212388 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.212199    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 21:37:39.212623 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.212458    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zlst6\""
Oct 30 21:37:39.212623 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.212511    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Oct 30 21:37:39.212623 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.212538    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Oct 30 21:37:39.212623 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.212565    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 21:37:39.212623 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.212599    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-ktl9g\""
Oct 30 21:37:39.212827 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.212673    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 21:37:39.212827 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.212750    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-hcvt7\""
Oct 30 21:37:39.212827 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.212781    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hkmjr\""
Oct 30 21:37:39.212827 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.212797    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 21:37:39.212827 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.212809    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 21:37:39.213039 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.212860    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-lt5wb\""
Oct 30 21:37:39.213039 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.212904    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-j5bph\""
Oct 30 21:37:39.213039 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.212928    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 21:37:39.213039 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.212944    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 21:37:39.213039 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.212956    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:37:39.213039 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.212962    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 21:37:39.213039 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.212973    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gfsxs\""
Oct 30 21:37:39.213039 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.212976    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-phtlh\""
Oct 30 21:37:39.213039 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.213024    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cb4gr\""
Oct 30 21:37:39.213304 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.213054    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 21:37:39.222454 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.222432    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-crio-socket\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:39.222529 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.222462    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a099b798-61cd-4306-85ba-85c7deb25a18-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-5lpq9\" (UID: \"a099b798-61cd-4306-85ba-85c7deb25a18\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-5lpq9"
Oct 30 21:37:39.222529 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.222478    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-etc-sysctl-conf\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:37:39.222529 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.222500    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/4086d1bd-6e47-4d50-a7f9-5be23d123d6e-snapshots\") pod \"insights-operator-7c4986d449-pf9px\" (UID: \"4086d1bd-6e47-4d50-a7f9-5be23d123d6e\") " pod="openshift-insights/insights-operator-7c4986d449-pf9px"
Oct 30 21:37:39.222647 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.222592    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/4086d1bd-6e47-4d50-a7f9-5be23d123d6e-snapshots\") pod \"insights-operator-7c4986d449-pf9px\" (UID: \"4086d1bd-6e47-4d50-a7f9-5be23d123d6e\") " pod="openshift-insights/insights-operator-7c4986d449-pf9px"
Oct 30 21:37:39.222647 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.222629    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cb8bde01-2bc6-4301-aa2f-2858158bd54e-cni-binary-copy\") pod \"multus-additional-cni-plugins-4frz9\" (UID: \"cb8bde01-2bc6-4301-aa2f-2858158bd54e\") " pod="openshift-multus/multus-additional-cni-plugins-4frz9"
Oct 30 21:37:39.222746 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.222668    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cb8bde01-2bc6-4301-aa2f-2858158bd54e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4frz9\" (UID: \"cb8bde01-2bc6-4301-aa2f-2858158bd54e\") " pod="openshift-multus/multus-additional-cni-plugins-4frz9"
Oct 30 21:37:39.222746 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.222702    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-574vn\" (UniqueName: \"kubernetes.io/projected/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-kube-api-access-574vn\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:37:39.222844 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.222762    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6d9d43c2-40da-47a8-adb4-d3ba378d77d9-konnectivity-ca\") pod \"konnectivity-agent-4vb42\" (UID: \"6d9d43c2-40da-47a8-adb4-d3ba378d77d9\") " pod="kube-system/konnectivity-agent-4vb42"
Oct 30 21:37:39.222844 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.222790    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b7002d6d-834c-4bdc-907e-805edd577a40-config\") pod \"console-operator-7cd56b8dbc-7tf9f\" (UID: \"b7002d6d-834c-4bdc-907e-805edd577a40\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f"
Oct 30 21:37:39.222844 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.222816    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0dca73b7-6526-48ac-a76e-70e441debc00-registry-certificates\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:39.222983 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.222854    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cb8bde01-2bc6-4301-aa2f-2858158bd54e-cni-binary-copy\") pod \"multus-additional-cni-plugins-4frz9\" (UID: \"cb8bde01-2bc6-4301-aa2f-2858158bd54e\") " pod="openshift-multus/multus-additional-cni-plugins-4frz9"
Oct 30 21:37:39.222983 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.222859    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-device-dir\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:37:39.222983 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.222901    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e0535562-96d7-4fc4-a5d5-ad4cc785080e-default-certificate\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:39.222983 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.222928    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlggr\" (UniqueName: \"kubernetes.io/projected/51e670da-3db6-412b-8753-8606d7a20fe3-kube-api-access-mlggr\") pod \"network-check-source-6f78d59f99-58v4l\" (UID: \"51e670da-3db6-412b-8753-8606d7a20fe3\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-58v4l"
Oct 30 21:37:39.222983 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.222949    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-installation-pull-secrets\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:39.223201 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.223000    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-run-systemd\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.223201 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.223034    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-host-var-lib-cni-multus\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.223201 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.223002    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0dca73b7-6526-48ac-a76e-70e441debc00-registry-certificates\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:39.223201 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.223061    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfcrl\" (UniqueName: \"kubernetes.io/projected/07b45ded-5ba8-49dc-b6ab-c170ca47af9b-kube-api-access-nfcrl\") pod \"iptables-alerter-2wb5p\" (UID: \"07b45ded-5ba8-49dc-b6ab-c170ca47af9b\") " pod="openshift-network-operator/iptables-alerter-2wb5p"
Oct 30 21:37:39.223201 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.223088    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b7002d6d-834c-4bdc-907e-805edd577a40-config\") pod \"console-operator-7cd56b8dbc-7tf9f\" (UID: \"b7002d6d-834c-4bdc-907e-805edd577a40\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f"
Oct 30 21:37:39.223201 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.223099    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-etc-sysctl-d\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:37:39.223201 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.223138    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6d9d43c2-40da-47a8-adb4-d3ba378d77d9-konnectivity-ca\") pod \"konnectivity-agent-4vb42\" (UID: \"6d9d43c2-40da-47a8-adb4-d3ba378d77d9\") " pod="kube-system/konnectivity-agent-4vb42"
Oct 30 21:37:39.223201 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.223145    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0dca73b7-6526-48ac-a76e-70e441debc00-installation-pull-secrets\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:39.223201 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.223162    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-installation-pull-secrets\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:39.223201 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.223167    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgdlc\" (UniqueName: \"kubernetes.io/projected/1dac48d8-aefb-4ff4-9213-b13def641c00-kube-api-access-kgdlc\") pod \"kas-connection-checker-685895c7c9-tlfjh\" (UID: \"1dac48d8-aefb-4ff4-9213-b13def641c00\") " pod="kube-system/kas-connection-checker-685895c7c9-tlfjh"
Oct 30 21:37:39.223647 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.223209    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-multus-socket-dir-parent\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.223647 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.223253    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-run-ovn-kubernetes\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.223647 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.223319    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kh4k2\" (UniqueName: \"kubernetes.io/projected/65827025-dc76-42d8-be9f-bdfed6b28429-kube-api-access-kh4k2\") pod \"cluster-monitoring-operator-884576c57-wltlg\" (UID: \"65827025-dc76-42d8-be9f-bdfed6b28429\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wltlg"
Oct 30 21:37:39.223647 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.223320    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e0535562-96d7-4fc4-a5d5-ad4cc785080e-default-certificate\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:39.223647 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.223346    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0dca73b7-6526-48ac-a76e-70e441debc00-installation-pull-secrets\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:39.223647 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.223354    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/be4fa9bf-41e0-4b26-9457-f9815a36d5b5-cert\") pod \"ingress-canary-rqxfc\" (UID: \"be4fa9bf-41e0-4b26-9457-f9815a36d5b5\") " pod="openshift-ingress-canary/ingress-canary-rqxfc"
Oct 30 21:37:39.223647 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.223374    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e615be3-41d7-4f82-90d7-7ce71ed9190b-original-pull-secret\") pod \"global-pull-secret-syncer-z9h2r\" (UID: \"6e615be3-41d7-4f82-90d7-7ce71ed9190b\") " pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:37:39.223647 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.223391    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-etc-kubernetes\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.223647 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.223423    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-etc-systemd\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:37:39.223647 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.223441    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/457fd859-0880-43f8-b712-2fd90c9debb0-etc-tuned\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:37:39.223647 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.223480    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-certificates\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:39.223647 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.223510    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-slash\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.223647 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.223525    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/be4fa9bf-41e0-4b26-9457-f9815a36d5b5-cert\") pod \"ingress-canary-rqxfc\" (UID: \"be4fa9bf-41e0-4b26-9457-f9815a36d5b5\") " pod="openshift-ingress-canary/ingress-canary-rqxfc"
Oct 30 21:37:39.223647 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.223530    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmxkp\" (UniqueName: \"kubernetes.io/projected/4086d1bd-6e47-4d50-a7f9-5be23d123d6e-kube-api-access-qmxkp\") pod \"insights-operator-7c4986d449-pf9px\" (UID: \"4086d1bd-6e47-4d50-a7f9-5be23d123d6e\") " pod="openshift-insights/insights-operator-7c4986d449-pf9px"
Oct 30 21:37:39.224332 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.223657    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5crvk\" (UniqueName: \"kubernetes.io/projected/872bccf3-b52e-4c52-9f5e-0487b4957b50-kube-api-access-5crvk\") pod \"dns-default-6q6pm\" (UID: \"872bccf3-b52e-4c52-9f5e-0487b4957b50\") " pod="openshift-dns/dns-default-6q6pm"
Oct 30 21:37:39.224332 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.223692    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrvv4\" (UniqueName: \"kubernetes.io/projected/909906d5-5049-4742-9b5a-b9c03ed5cc39-kube-api-access-vrvv4\") pod \"node-resolver-dbpfd\" (UID: \"909906d5-5049-4742-9b5a-b9c03ed5cc39\") " pod="openshift-dns/node-resolver-dbpfd"
Oct 30 21:37:39.224332 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.223719    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cb8bde01-2bc6-4301-aa2f-2858158bd54e-cnibin\") pod \"multus-additional-cni-plugins-4frz9\" (UID: \"cb8bde01-2bc6-4301-aa2f-2858158bd54e\") " pod="openshift-multus/multus-additional-cni-plugins-4frz9"
Oct 30 21:37:39.224332 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.223748    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cb8bde01-2bc6-4301-aa2f-2858158bd54e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4frz9\" (UID: \"cb8bde01-2bc6-4301-aa2f-2858158bd54e\") " pod="openshift-multus/multus-additional-cni-plugins-4frz9"
Oct 30 21:37:39.224332 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.223776    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cb8bde01-2bc6-4301-aa2f-2858158bd54e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4frz9\" (UID: \"cb8bde01-2bc6-4301-aa2f-2858158bd54e\") " pod="openshift-multus/multus-additional-cni-plugins-4frz9"
Oct 30 21:37:39.224332 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.223820    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-host-run-k8s-cni-cncf-io\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.224332 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.223860    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-run-netns\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.224332 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.223891    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-cni-netd\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.224332 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.223940    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.224332 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.223977    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb8bde01-2bc6-4301-aa2f-2858158bd54e-system-cni-dir\") pod \"multus-additional-cni-plugins-4frz9\" (UID: \"cb8bde01-2bc6-4301-aa2f-2858158bd54e\") " pod="openshift-multus/multus-additional-cni-plugins-4frz9"
Oct 30 21:37:39.224332 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.223991    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cb8bde01-2bc6-4301-aa2f-2858158bd54e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4frz9\" (UID: \"cb8bde01-2bc6-4301-aa2f-2858158bd54e\") " pod="openshift-multus/multus-additional-cni-plugins-4frz9"
Oct 30 21:37:39.224332 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.223537    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/457fd859-0880-43f8-b712-2fd90c9debb0-etc-tuned\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:37:39.224332 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.224029    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:39.224332 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.224056    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fb70ad86-4b45-4a53-a3cb-1f6b59640708-config\") pod \"service-ca-operator-544d749c8c-qqxdq\" (UID: \"fb70ad86-4b45-4a53-a3cb-1f6b59640708\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-qqxdq"
Oct 30 21:37:39.224332 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.224063    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cb8bde01-2bc6-4301-aa2f-2858158bd54e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4frz9\" (UID: \"cb8bde01-2bc6-4301-aa2f-2858158bd54e\") " pod="openshift-multus/multus-additional-cni-plugins-4frz9"
Oct 30 21:37:39.224332 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.224086    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7976h\" (UniqueName: \"kubernetes.io/projected/fb70ad86-4b45-4a53-a3cb-1f6b59640708-kube-api-access-7976h\") pod \"service-ca-operator-544d749c8c-qqxdq\" (UID: \"fb70ad86-4b45-4a53-a3cb-1f6b59640708\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-qqxdq"
Oct 30 21:37:39.224332 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.224134    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-certificates\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:39.224332 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.224171    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-registry-tls\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:39.224332 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.224203    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8hlw\" (UniqueName: \"kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-kube-api-access-g8hlw\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:39.224332 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.224231    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/726700d7-c252-479e-abdd-18c0573102d1-cni-binary-copy\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.225117 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.224351    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2360fddc-52a5-42a5-82ca-a05aff84d217-ovnkube-config\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.225117 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.224377    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn8jz\" (UniqueName: \"kubernetes.io/projected/37186e06-8d56-4c61-ba10-e0408d2e2fa4-kube-api-access-zn8jz\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-jgspb\" (UID: \"37186e06-8d56-4c61-ba10-e0408d2e2fa4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-jgspb"
Oct 30 21:37:39.225117 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.224405    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-etc-modprobe-d\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:37:39.225117 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.224448    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lnqk\" (UniqueName: \"kubernetes.io/projected/457fd859-0880-43f8-b712-2fd90c9debb0-kube-api-access-6lnqk\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:37:39.225117 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.224475    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-host-var-lib-cni-bin\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.225117 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.224513    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-hostroot\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.225117 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.224546    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-host-run-multus-certs\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.225117 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.224572    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/726700d7-c252-479e-abdd-18c0573102d1-cni-binary-copy\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.225117 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.224574    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-kubelet\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.225117 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.224587    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fb70ad86-4b45-4a53-a3cb-1f6b59640708-config\") pod \"service-ca-operator-544d749c8c-qqxdq\" (UID: \"fb70ad86-4b45-4a53-a3cb-1f6b59640708\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-qqxdq"
Oct 30 21:37:39.225117 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.224598    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2360fddc-52a5-42a5-82ca-a05aff84d217-ovnkube-config\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.225117 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.224632    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-systemd-units\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.225117 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.224668    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-etc-kubernetes\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:37:39.225117 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.224697    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-multus-cni-dir\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.225117 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.224728    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kbb8\" (UniqueName: \"kubernetes.io/projected/d54af961-c593-4707-9cec-1069000bce50-kube-api-access-7kbb8\") pod \"kas-connection-checker-685895c7c9-nvwdl\" (UID: \"d54af961-c593-4707-9cec-1069000bce50\") " pod="kube-system/kas-connection-checker-685895c7c9-nvwdl"
Oct 30 21:37:39.225117 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.224743    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-etc-sysconfig\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:37:39.225117 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.224759    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5cf90653-8f33-4bb2-b622-1dae9144a9fe-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-q2pns\" (UID: \"5cf90653-8f33-4bb2-b622-1dae9144a9fe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q2pns"
Oct 30 21:37:39.225117 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.224775    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e0535562-96d7-4fc4-a5d5-ad4cc785080e-stats-auth\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:39.225117 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.224931    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/457fd859-0880-43f8-b712-2fd90c9debb0-tmp\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:37:39.225117 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.224940    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e0535562-96d7-4fc4-a5d5-ad4cc785080e-stats-auth\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:39.225117 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.224960    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/872bccf3-b52e-4c52-9f5e-0487b4957b50-metrics-tls\") pod \"dns-default-6q6pm\" (UID: \"872bccf3-b52e-4c52-9f5e-0487b4957b50\") " pod="openshift-dns/dns-default-6q6pm"
Oct 30 21:37:39.225117 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.224986    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-multus-conf-dir\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.225117 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.224999    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/457fd859-0880-43f8-b712-2fd90c9debb0-tmp\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:37:39.225117 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.225011    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/726700d7-c252-479e-abdd-18c0573102d1-multus-daemon-config\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.225117 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.225040    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/07b45ded-5ba8-49dc-b6ab-c170ca47af9b-host-slash\") pod \"iptables-alerter-2wb5p\" (UID: \"07b45ded-5ba8-49dc-b6ab-c170ca47af9b\") " pod="openshift-network-operator/iptables-alerter-2wb5p"
Oct 30 21:37:39.226150 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.225149    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/726700d7-c252-479e-abdd-18c0573102d1-multus-daemon-config\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.226150 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.225153    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-image-registry-private-configuration\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:39.226150 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.225187    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-ca-trust-extracted\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:39.226150 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.225228    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4086d1bd-6e47-4d50-a7f9-5be23d123d6e-tmp\") pod \"insights-operator-7c4986d449-pf9px\" (UID: \"4086d1bd-6e47-4d50-a7f9-5be23d123d6e\") " pod="openshift-insights/insights-operator-7c4986d449-pf9px"
Oct 30 21:37:39.226150 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.225274    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:37:39.226150 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.225291    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-ca-trust-extracted\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:39.226150 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.225299    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/909906d5-5049-4742-9b5a-b9c03ed5cc39-hosts-file\") pod \"node-resolver-dbpfd\" (UID: \"909906d5-5049-4742-9b5a-b9c03ed5cc39\") " pod="openshift-dns/node-resolver-dbpfd"
Oct 30 21:37:39.226150 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.225307    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4086d1bd-6e47-4d50-a7f9-5be23d123d6e-tmp\") pod \"insights-operator-7c4986d449-pf9px\" (UID: \"4086d1bd-6e47-4d50-a7f9-5be23d123d6e\") " pod="openshift-insights/insights-operator-7c4986d449-pf9px"
Oct 30 21:37:39.226150 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.225323    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-host-var-lib-kubelet\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.226150 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.225339    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-image-registry-private-configuration\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:39.226150 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.225350    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-etc-openvswitch\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.226150 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.225403    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7lhk\" (UniqueName: \"kubernetes.io/projected/2360fddc-52a5-42a5-82ca-a05aff84d217-kube-api-access-v7lhk\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.226150 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.225522    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-socket-dir\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:37:39.226150 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.225553    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/5cf90653-8f33-4bb2-b622-1dae9144a9fe-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-q2pns\" (UID: \"5cf90653-8f33-4bb2-b622-1dae9144a9fe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q2pns"
Oct 30 21:37:39.226150 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.225584    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxmv7\" (UniqueName: \"kubernetes.io/projected/9eefa84c-5304-4c2a-89cf-3304097ecf42-kube-api-access-gxmv7\") pod \"network-check-target-wpc6f\" (UID: \"9eefa84c-5304-4c2a-89cf-3304097ecf42\") " pod="openshift-network-diagnostics/network-check-target-wpc6f"
Oct 30 21:37:39.226150 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.225607    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-host\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:37:39.226150 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.225626    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-sys-fs\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:37:39.226150 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.225643    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-host-run-netns\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.226150 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.225671    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxsh7\" (UniqueName: \"kubernetes.io/projected/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-kube-api-access-jxsh7\") pod \"network-metrics-daemon-l9kxl\" (UID: \"7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2\") " pod="openshift-multus/network-metrics-daemon-l9kxl"
Oct 30 21:37:39.226150 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.225697    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4qb5\" (UniqueName: \"kubernetes.io/projected/b7002d6d-834c-4bdc-907e-805edd577a40-kube-api-access-c4qb5\") pod \"console-operator-7cd56b8dbc-7tf9f\" (UID: \"b7002d6d-834c-4bdc-907e-805edd577a40\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f"
Oct 30 21:37:39.226150 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.225716    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/948affe7-aaf8-4d3e-a515-ed9552d19d90-host\") pod \"node-ca-grk5c\" (UID: \"948affe7-aaf8-4d3e-a515-ed9552d19d90\") " pod="openshift-image-registry/node-ca-grk5c"
Oct 30 21:37:39.226150 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.225771    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/65827025-dc76-42d8-be9f-bdfed6b28429-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-wltlg\" (UID: \"65827025-dc76-42d8-be9f-bdfed6b28429\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wltlg"
Oct 30 21:37:39.226150 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.225790    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/5cf90653-8f33-4bb2-b622-1dae9144a9fe-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-q2pns\" (UID: \"5cf90653-8f33-4bb2-b622-1dae9144a9fe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q2pns"
Oct 30 21:37:39.226150 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.225807    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fb70ad86-4b45-4a53-a3cb-1f6b59640708-serving-cert\") pod \"service-ca-operator-544d749c8c-qqxdq\" (UID: \"fb70ad86-4b45-4a53-a3cb-1f6b59640708\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-qqxdq"
Oct 30 21:37:39.226150 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.225836    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/872bccf3-b52e-4c52-9f5e-0487b4957b50-tmp-dir\") pod \"dns-default-6q6pm\" (UID: \"872bccf3-b52e-4c52-9f5e-0487b4957b50\") " pod="openshift-dns/dns-default-6q6pm"
Oct 30 21:37:39.226150 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.225852    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/909906d5-5049-4742-9b5a-b9c03ed5cc39-tmp-dir\") pod \"node-resolver-dbpfd\" (UID: \"909906d5-5049-4742-9b5a-b9c03ed5cc39\") " pod="openshift-dns/node-resolver-dbpfd"
Oct 30 21:37:39.226150 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.225867    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-system-cni-dir\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.226150 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.225881    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-log-socket\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.226150 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.225896    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/872bccf3-b52e-4c52-9f5e-0487b4957b50-tmp-dir\") pod \"dns-default-6q6pm\" (UID: \"872bccf3-b52e-4c52-9f5e-0487b4957b50\") " pod="openshift-dns/dns-default-6q6pm"
Oct 30 21:37:39.226150 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.225899    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbs5d\" (UniqueName: \"kubernetes.io/projected/5cf90653-8f33-4bb2-b622-1dae9144a9fe-kube-api-access-xbs5d\") pod \"cluster-samples-operator-6ddb6c8b94-q2pns\" (UID: \"5cf90653-8f33-4bb2-b622-1dae9144a9fe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q2pns"
Oct 30 21:37:39.226150 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.225914    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/909906d5-5049-4742-9b5a-b9c03ed5cc39-tmp-dir\") pod \"node-resolver-dbpfd\" (UID: \"909906d5-5049-4742-9b5a-b9c03ed5cc39\") " pod="openshift-dns/node-resolver-dbpfd"
Oct 30 21:37:39.226150 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.225936    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fb70ad86-4b45-4a53-a3cb-1f6b59640708-serving-cert\") pod \"service-ca-operator-544d749c8c-qqxdq\" (UID: \"fb70ad86-4b45-4a53-a3cb-1f6b59640708\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-qqxdq"
Oct 30 21:37:39.226150 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.226022    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phfjb\" (UniqueName: \"kubernetes.io/projected/5f5757a0-0a11-4637-9aa5-4e283792017d-kube-api-access-phfjb\") pod \"kas-connection-checker-685895c7c9-px9dx\" (UID: \"5f5757a0-0a11-4637-9aa5-4e283792017d\") " pod="kube-system/kas-connection-checker-685895c7c9-px9dx"
Oct 30 21:37:39.226150 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.226053    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/37186e06-8d56-4c61-ba10-e0408d2e2fa4-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-jgspb\" (UID: \"37186e06-8d56-4c61-ba10-e0408d2e2fa4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-jgspb"
Oct 30 21:37:39.227619 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.226071    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-sys\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:37:39.227619 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.226395    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-bound-sa-token\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:39.227619 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.226453    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0dca73b7-6526-48ac-a76e-70e441debc00-trusted-ca\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:39.227619 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.226488    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0dca73b7-6526-48ac-a76e-70e441debc00-image-registry-private-configuration\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:39.227619 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.226522    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpmml\" (UniqueName: \"kubernetes.io/projected/be4fa9bf-41e0-4b26-9457-f9815a36d5b5-kube-api-access-vpmml\") pod \"ingress-canary-rqxfc\" (UID: \"be4fa9bf-41e0-4b26-9457-f9815a36d5b5\") " pod="openshift-ingress-canary/ingress-canary-rqxfc"
Oct 30 21:37:39.227619 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.226553    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-var-lib-openvswitch\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.227619 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.226576    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/37186e06-8d56-4c61-ba10-e0408d2e2fa4-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-jgspb\" (UID: \"37186e06-8d56-4c61-ba10-e0408d2e2fa4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-jgspb"
Oct 30 21:37:39.227619 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.226749    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-var-lib-kubelet\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:37:39.227619 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.226783    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-node-log\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.227619 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.226816    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2360fddc-52a5-42a5-82ca-a05aff84d217-ovnkube-script-lib\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.227619 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.226842    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0dca73b7-6526-48ac-a76e-70e441debc00-image-registry-private-configuration\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:39.227619 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.226845    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-bound-sa-token\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:39.227619 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.226903    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-registration-dir\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:37:39.227619 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.226940    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-data-volume\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:39.227619 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.226972    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:39.227619 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.227022    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a099b798-61cd-4306-85ba-85c7deb25a18-nginx-conf\") pod \"networking-console-plugin-86c478bb88-5lpq9\" (UID: \"a099b798-61cd-4306-85ba-85c7deb25a18\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-5lpq9"
Oct 30 21:37:39.227619 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.227052    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-run-ovn\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.227619 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.227226    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-data-volume\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:39.227619 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.227297    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a099b798-61cd-4306-85ba-85c7deb25a18-nginx-conf\") pod \"networking-console-plugin-86c478bb88-5lpq9\" (UID: \"a099b798-61cd-4306-85ba-85c7deb25a18\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-5lpq9"
Oct 30 21:37:39.227619 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.227402    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0535562-96d7-4fc4-a5d5-ad4cc785080e-metrics-certs\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:39.227619 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.227440    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/37186e06-8d56-4c61-ba10-e0408d2e2fa4-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-jgspb\" (UID: \"37186e06-8d56-4c61-ba10-e0408d2e2fa4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-jgspb"
Oct 30 21:37:39.227619 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.227473    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-trusted-ca\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:39.227619 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.227500    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4086d1bd-6e47-4d50-a7f9-5be23d123d6e-serving-cert\") pod \"insights-operator-7c4986d449-pf9px\" (UID: \"4086d1bd-6e47-4d50-a7f9-5be23d123d6e\") " pod="openshift-insights/insights-operator-7c4986d449-pf9px"
Oct 30 21:37:39.227619 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.227526    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2360fddc-52a5-42a5-82ca-a05aff84d217-ovnkube-script-lib\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.227619 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.227531    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wv5hx\" (UniqueName: \"kubernetes.io/projected/948affe7-aaf8-4d3e-a515-ed9552d19d90-kube-api-access-wv5hx\") pod \"node-ca-grk5c\" (UID: \"948affe7-aaf8-4d3e-a515-ed9552d19d90\") " pod="openshift-image-registry/node-ca-grk5c"
Oct 30 21:37:39.227619 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.227567    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/65827025-dc76-42d8-be9f-bdfed6b28429-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-wltlg\" (UID: \"65827025-dc76-42d8-be9f-bdfed6b28429\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wltlg"
Oct 30 21:37:39.227619 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.227597    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-run\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:37:39.227619 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.227626    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-tls\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:39.228752 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.227673    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4086d1bd-6e47-4d50-a7f9-5be23d123d6e-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-pf9px\" (UID: \"4086d1bd-6e47-4d50-a7f9-5be23d123d6e\") " pod="openshift-insights/insights-operator-7c4986d449-pf9px"
Oct 30 21:37:39.228752 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.227708    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-etc-selinux\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:37:39.228752 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.227713    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/37186e06-8d56-4c61-ba10-e0408d2e2fa4-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-jgspb\" (UID: \"37186e06-8d56-4c61-ba10-e0408d2e2fa4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-jgspb"
Oct 30 21:37:39.228752 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.227740    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-metrics-certs\") pod \"network-metrics-daemon-l9kxl\" (UID: \"7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2\") " pod="openshift-multus/network-metrics-daemon-l9kxl"
Oct 30 21:37:39.228752 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.227770    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/872bccf3-b52e-4c52-9f5e-0487b4957b50-config-volume\") pod \"dns-default-6q6pm\" (UID: \"872bccf3-b52e-4c52-9f5e-0487b4957b50\") " pod="openshift-dns/dns-default-6q6pm"
Oct 30 21:37:39.228752 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.227801    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-cnibin\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.228752 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.227830    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2360fddc-52a5-42a5-82ca-a05aff84d217-env-overrides\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.228752 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.227854    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-lib-modules\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:37:39.228752 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.227883    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmlvz\" (UniqueName: \"kubernetes.io/projected/cb8bde01-2bc6-4301-aa2f-2858158bd54e-kube-api-access-qmlvz\") pod \"multus-additional-cni-plugins-4frz9\" (UID: \"cb8bde01-2bc6-4301-aa2f-2858158bd54e\") " pod="openshift-multus/multus-additional-cni-plugins-4frz9"
Oct 30 21:37:39.228752 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.227913    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zgrn\" (UniqueName: \"kubernetes.io/projected/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-kube-api-access-2zgrn\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:39.228752 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.227942    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-cni-bin\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.228752 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.228072    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4086d1bd-6e47-4d50-a7f9-5be23d123d6e-service-ca-bundle\") pod \"insights-operator-7c4986d449-pf9px\" (UID: \"4086d1bd-6e47-4d50-a7f9-5be23d123d6e\") " pod="openshift-insights/insights-operator-7c4986d449-pf9px"
Oct 30 21:37:39.228752 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.228260    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4086d1bd-6e47-4d50-a7f9-5be23d123d6e-service-ca-bundle\") pod \"insights-operator-7c4986d449-pf9px\" (UID: \"4086d1bd-6e47-4d50-a7f9-5be23d123d6e\") " pod="openshift-insights/insights-operator-7c4986d449-pf9px"
Oct 30 21:37:39.228752 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.228264    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/872bccf3-b52e-4c52-9f5e-0487b4957b50-config-volume\") pod \"dns-default-6q6pm\" (UID: \"872bccf3-b52e-4c52-9f5e-0487b4957b50\") " pod="openshift-dns/dns-default-6q6pm"
Oct 30 21:37:39.228752 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.228424    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2360fddc-52a5-42a5-82ca-a05aff84d217-env-overrides\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.228752 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.228683    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4086d1bd-6e47-4d50-a7f9-5be23d123d6e-serving-cert\") pod \"insights-operator-7c4986d449-pf9px\" (UID: \"4086d1bd-6e47-4d50-a7f9-5be23d123d6e\") " pod="openshift-insights/insights-operator-7c4986d449-pf9px"
Oct 30 21:37:39.228752 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.228714    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0dca73b7-6526-48ac-a76e-70e441debc00-ca-trust-extracted\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:39.229548 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.228780    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/948affe7-aaf8-4d3e-a515-ed9552d19d90-serviceca\") pod \"node-ca-grk5c\" (UID: \"948affe7-aaf8-4d3e-a515-ed9552d19d90\") " pod="openshift-image-registry/node-ca-grk5c"
Oct 30 21:37:39.229548 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.228839    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6e615be3-41d7-4f82-90d7-7ce71ed9190b-kubelet-config\") pod \"global-pull-secret-syncer-z9h2r\" (UID: \"6e615be3-41d7-4f82-90d7-7ce71ed9190b\") " pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:37:39.229548 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.228881    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f26sz\" (UniqueName: \"kubernetes.io/projected/e0535562-96d7-4fc4-a5d5-ad4cc785080e-kube-api-access-f26sz\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:39.229548 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.228913    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/07b45ded-5ba8-49dc-b6ab-c170ca47af9b-iptables-alerter-script\") pod \"iptables-alerter-2wb5p\" (UID: \"07b45ded-5ba8-49dc-b6ab-c170ca47af9b\") " pod="openshift-network-operator/iptables-alerter-2wb5p"
Oct 30 21:37:39.229548 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.229430    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/948affe7-aaf8-4d3e-a515-ed9552d19d90-serviceca\") pod \"node-ca-grk5c\" (UID: \"948affe7-aaf8-4d3e-a515-ed9552d19d90\") " pod="openshift-image-registry/node-ca-grk5c"
Oct 30 21:37:39.229548 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.229511    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0dca73b7-6526-48ac-a76e-70e441debc00-ca-trust-extracted\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:39.229548 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.229547    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6e615be3-41d7-4f82-90d7-7ce71ed9190b-dbus\") pod \"global-pull-secret-syncer-z9h2r\" (UID: \"6e615be3-41d7-4f82-90d7-7ce71ed9190b\") " pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:37:39.229841 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.229586    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4xm6\" (UniqueName: \"kubernetes.io/projected/726700d7-c252-479e-abdd-18c0573102d1-kube-api-access-z4xm6\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.229841 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.229616    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b7002d6d-834c-4bdc-907e-805edd577a40-serving-cert\") pod \"console-operator-7cd56b8dbc-7tf9f\" (UID: \"b7002d6d-834c-4bdc-907e-805edd577a40\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f"
Oct 30 21:37:39.229841 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.229645    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d49lh\" (UniqueName: \"kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-kube-api-access-d49lh\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:39.229841 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.229676    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-run-openvswitch\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.229841 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.229708    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zs5q4\" (UniqueName: \"kubernetes.io/projected/32888534-d682-4428-90d9-03c858ce5ab8-kube-api-access-zs5q4\") pod \"volume-data-source-validator-58c9f5f46d-7lnsg\" (UID: \"32888534-d682-4428-90d9-03c858ce5ab8\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-7lnsg"
Oct 30 21:37:39.229841 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.229736    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0535562-96d7-4fc4-a5d5-ad4cc785080e-service-ca-bundle\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:39.229841 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.229767    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6d9d43c2-40da-47a8-adb4-d3ba378d77d9-agent-certs\") pod \"konnectivity-agent-4vb42\" (UID: \"6d9d43c2-40da-47a8-adb4-d3ba378d77d9\") " pod="kube-system/konnectivity-agent-4vb42"
Oct 30 21:37:39.229841 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.229795    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b7002d6d-834c-4bdc-907e-805edd577a40-trusted-ca\") pod \"console-operator-7cd56b8dbc-7tf9f\" (UID: \"b7002d6d-834c-4bdc-907e-805edd577a40\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f"
Oct 30 21:37:39.229841 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.229825    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2360fddc-52a5-42a5-82ca-a05aff84d217-ovn-node-metrics-cert\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.230212 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.230075    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2360fddc-52a5-42a5-82ca-a05aff84d217-ovn-node-metrics-cert\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.230723 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.230700    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b7002d6d-834c-4bdc-907e-805edd577a40-serving-cert\") pod \"console-operator-7cd56b8dbc-7tf9f\" (UID: \"b7002d6d-834c-4bdc-907e-805edd577a40\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f"
Oct 30 21:37:39.254468 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.254436    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tsx8f\""
Oct 30 21:37:39.254598 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.254501    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-x7gkt\""
Oct 30 21:37:39.254882 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.254864    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 21:37:39.254940 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.254928    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-b6pf6\""
Oct 30 21:37:39.254972 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.254945    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-v9xch\""
Oct 30 21:37:39.254972 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.254945    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 21:37:39.257970 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.257955    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 21:37:39.258193 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.258180    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Oct 30 21:37:39.258228 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.258213    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xlcqp\""
Oct 30 21:37:39.258228 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.258214    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-p5bth\""
Oct 30 21:37:39.259852 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.259831    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/07b45ded-5ba8-49dc-b6ab-c170ca47af9b-iptables-alerter-script\") pod \"iptables-alerter-2wb5p\" (UID: \"07b45ded-5ba8-49dc-b6ab-c170ca47af9b\") " pod="openshift-network-operator/iptables-alerter-2wb5p"
Oct 30 21:37:39.260188 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.260173    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 30 21:37:39.260278 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.260212    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Oct 30 21:37:39.260317 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.260295    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Oct 30 21:37:39.261470 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.261447    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6d9d43c2-40da-47a8-adb4-d3ba378d77d9-agent-certs\") pod \"konnectivity-agent-4vb42\" (UID: \"6d9d43c2-40da-47a8-adb4-d3ba378d77d9\") " pod="kube-system/konnectivity-agent-4vb42"
Oct 30 21:37:39.261855 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.261840    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b7002d6d-834c-4bdc-907e-805edd577a40-trusted-ca\") pod \"console-operator-7cd56b8dbc-7tf9f\" (UID: \"b7002d6d-834c-4bdc-907e-805edd577a40\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f"
Oct 30 21:37:39.264597 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.264580    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e615be3-41d7-4f82-90d7-7ce71ed9190b-original-pull-secret\") pod \"global-pull-secret-syncer-z9h2r\" (UID: \"6e615be3-41d7-4f82-90d7-7ce71ed9190b\") " pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:37:39.266759 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.266742    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/65827025-dc76-42d8-be9f-bdfed6b28429-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-wltlg\" (UID: \"65827025-dc76-42d8-be9f-bdfed6b28429\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wltlg"
Oct 30 21:37:39.266892 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.266877    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0dca73b7-6526-48ac-a76e-70e441debc00-trusted-ca\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:39.269143 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.269126    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4086d1bd-6e47-4d50-a7f9-5be23d123d6e-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-pf9px\" (UID: \"4086d1bd-6e47-4d50-a7f9-5be23d123d6e\") " pod="openshift-insights/insights-operator-7c4986d449-pf9px"
Oct 30 21:37:39.269180 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.269150    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-trusted-ca\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:39.283381 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.283356    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgdlc\" (UniqueName: \"kubernetes.io/projected/1dac48d8-aefb-4ff4-9213-b13def641c00-kube-api-access-kgdlc\") pod \"kas-connection-checker-685895c7c9-tlfjh\" (UID: \"1dac48d8-aefb-4ff4-9213-b13def641c00\") " pod="kube-system/kas-connection-checker-685895c7c9-tlfjh"
Oct 30 21:37:39.283461 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.283419    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kbb8\" (UniqueName: \"kubernetes.io/projected/d54af961-c593-4707-9cec-1069000bce50-kube-api-access-7kbb8\") pod \"kas-connection-checker-685895c7c9-nvwdl\" (UID: \"d54af961-c593-4707-9cec-1069000bce50\") " pod="kube-system/kas-connection-checker-685895c7c9-nvwdl"
Oct 30 21:37:39.283618 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.283602    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlggr\" (UniqueName: \"kubernetes.io/projected/51e670da-3db6-412b-8753-8606d7a20fe3-kube-api-access-mlggr\") pod \"network-check-source-6f78d59f99-58v4l\" (UID: \"51e670da-3db6-412b-8753-8606d7a20fe3\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-58v4l"
Oct 30 21:37:39.283850 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.283833    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmxkp\" (UniqueName: \"kubernetes.io/projected/4086d1bd-6e47-4d50-a7f9-5be23d123d6e-kube-api-access-qmxkp\") pod \"insights-operator-7c4986d449-pf9px\" (UID: \"4086d1bd-6e47-4d50-a7f9-5be23d123d6e\") " pod="openshift-insights/insights-operator-7c4986d449-pf9px"
Oct 30 21:37:39.285446 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.285429    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxsh7\" (UniqueName: \"kubernetes.io/projected/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-kube-api-access-jxsh7\") pod \"network-metrics-daemon-l9kxl\" (UID: \"7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2\") " pod="openshift-multus/network-metrics-daemon-l9kxl"
Oct 30 21:37:39.285927 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.285913    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-bound-sa-token\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:39.286712 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.286695    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrvv4\" (UniqueName: \"kubernetes.io/projected/909906d5-5049-4742-9b5a-b9c03ed5cc39-kube-api-access-vrvv4\") pod \"node-resolver-dbpfd\" (UID: \"909906d5-5049-4742-9b5a-b9c03ed5cc39\") " pod="openshift-dns/node-resolver-dbpfd"
Oct 30 21:37:39.288064 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.288043    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-574vn\" (UniqueName: \"kubernetes.io/projected/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-kube-api-access-574vn\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:37:39.288272 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.288252    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5crvk\" (UniqueName: \"kubernetes.io/projected/872bccf3-b52e-4c52-9f5e-0487b4957b50-kube-api-access-5crvk\") pod \"dns-default-6q6pm\" (UID: \"872bccf3-b52e-4c52-9f5e-0487b4957b50\") " pod="openshift-dns/dns-default-6q6pm"
Oct 30 21:37:39.288331 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.288317    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfcrl\" (UniqueName: \"kubernetes.io/projected/07b45ded-5ba8-49dc-b6ab-c170ca47af9b-kube-api-access-nfcrl\") pod \"iptables-alerter-2wb5p\" (UID: \"07b45ded-5ba8-49dc-b6ab-c170ca47af9b\") " pod="openshift-network-operator/iptables-alerter-2wb5p"
Oct 30 21:37:39.288868 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.288847    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8hlw\" (UniqueName: \"kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-kube-api-access-g8hlw\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:39.289232 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.289218    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpmml\" (UniqueName: \"kubernetes.io/projected/be4fa9bf-41e0-4b26-9457-f9815a36d5b5-kube-api-access-vpmml\") pod \"ingress-canary-rqxfc\" (UID: \"be4fa9bf-41e0-4b26-9457-f9815a36d5b5\") " pod="openshift-ingress-canary/ingress-canary-rqxfc"
Oct 30 21:37:39.290640 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.290624    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4qb5\" (UniqueName: \"kubernetes.io/projected/b7002d6d-834c-4bdc-907e-805edd577a40-kube-api-access-c4qb5\") pod \"console-operator-7cd56b8dbc-7tf9f\" (UID: \"b7002d6d-834c-4bdc-907e-805edd577a40\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f"
Oct 30 21:37:39.291752 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.291737    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmlvz\" (UniqueName: \"kubernetes.io/projected/cb8bde01-2bc6-4301-aa2f-2858158bd54e-kube-api-access-qmlvz\") pod \"multus-additional-cni-plugins-4frz9\" (UID: \"cb8bde01-2bc6-4301-aa2f-2858158bd54e\") " pod="openshift-multus/multus-additional-cni-plugins-4frz9"
Oct 30 21:37:39.292459 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.292443    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-bound-sa-token\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:39.293453 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.293433    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxmv7\" (UniqueName: \"kubernetes.io/projected/9eefa84c-5304-4c2a-89cf-3304097ecf42-kube-api-access-gxmv7\") pod \"network-check-target-wpc6f\" (UID: \"9eefa84c-5304-4c2a-89cf-3304097ecf42\") " pod="openshift-network-diagnostics/network-check-target-wpc6f"
Oct 30 21:37:39.293525 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.293512    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbs5d\" (UniqueName: \"kubernetes.io/projected/5cf90653-8f33-4bb2-b622-1dae9144a9fe-kube-api-access-xbs5d\") pod \"cluster-samples-operator-6ddb6c8b94-q2pns\" (UID: \"5cf90653-8f33-4bb2-b622-1dae9144a9fe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q2pns"
Oct 30 21:37:39.297338 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.297300    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zgrn\" (UniqueName: \"kubernetes.io/projected/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-kube-api-access-2zgrn\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:39.297485 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.297463    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7976h\" (UniqueName: \"kubernetes.io/projected/fb70ad86-4b45-4a53-a3cb-1f6b59640708-kube-api-access-7976h\") pod \"service-ca-operator-544d749c8c-qqxdq\" (UID: \"fb70ad86-4b45-4a53-a3cb-1f6b59640708\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-qqxdq"
Oct 30 21:37:39.297578 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.297557    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7lhk\" (UniqueName: \"kubernetes.io/projected/2360fddc-52a5-42a5-82ca-a05aff84d217-kube-api-access-v7lhk\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.297631 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.297594    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zn8jz\" (UniqueName: \"kubernetes.io/projected/37186e06-8d56-4c61-ba10-e0408d2e2fa4-kube-api-access-zn8jz\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-jgspb\" (UID: \"37186e06-8d56-4c61-ba10-e0408d2e2fa4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-jgspb"
Oct 30 21:37:39.297631 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.297626    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lnqk\" (UniqueName: \"kubernetes.io/projected/457fd859-0880-43f8-b712-2fd90c9debb0-kube-api-access-6lnqk\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:37:39.297730 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.297639    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kh4k2\" (UniqueName: \"kubernetes.io/projected/65827025-dc76-42d8-be9f-bdfed6b28429-kube-api-access-kh4k2\") pod \"cluster-monitoring-operator-884576c57-wltlg\" (UID: \"65827025-dc76-42d8-be9f-bdfed6b28429\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wltlg"
Oct 30 21:37:39.301012 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.300997    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phfjb\" (UniqueName: \"kubernetes.io/projected/5f5757a0-0a11-4637-9aa5-4e283792017d-kube-api-access-phfjb\") pod \"kas-connection-checker-685895c7c9-px9dx\" (UID: \"5f5757a0-0a11-4637-9aa5-4e283792017d\") " pod="kube-system/kas-connection-checker-685895c7c9-px9dx"
Oct 30 21:37:39.308629 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.308600    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f26sz\" (UniqueName: \"kubernetes.io/projected/e0535562-96d7-4fc4-a5d5-ad4cc785080e-kube-api-access-f26sz\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:39.308830 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.308812    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4xm6\" (UniqueName: \"kubernetes.io/projected/726700d7-c252-479e-abdd-18c0573102d1-kube-api-access-z4xm6\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.308981 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.308965    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zs5q4\" (UniqueName: \"kubernetes.io/projected/32888534-d682-4428-90d9-03c858ce5ab8-kube-api-access-zs5q4\") pod \"volume-data-source-validator-58c9f5f46d-7lnsg\" (UID: \"32888534-d682-4428-90d9-03c858ce5ab8\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-7lnsg"
Oct 30 21:37:39.311048 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.311032    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wv5hx\" (UniqueName: \"kubernetes.io/projected/948affe7-aaf8-4d3e-a515-ed9552d19d90-kube-api-access-wv5hx\") pod \"node-ca-grk5c\" (UID: \"948affe7-aaf8-4d3e-a515-ed9552d19d90\") " pod="openshift-image-registry/node-ca-grk5c"
Oct 30 21:37:39.311139 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.311124    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d49lh\" (UniqueName: \"kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-kube-api-access-d49lh\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:39.330398 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.330374    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6e615be3-41d7-4f82-90d7-7ce71ed9190b-kubelet-config\") pod \"global-pull-secret-syncer-z9h2r\" (UID: \"6e615be3-41d7-4f82-90d7-7ce71ed9190b\") " pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:37:39.330479 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.330402    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6e615be3-41d7-4f82-90d7-7ce71ed9190b-dbus\") pod \"global-pull-secret-syncer-z9h2r\" (UID: \"6e615be3-41d7-4f82-90d7-7ce71ed9190b\") " pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:37:39.330479 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.330419    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-run-openvswitch\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.330479 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.330434    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0535562-96d7-4fc4-a5d5-ad4cc785080e-service-ca-bundle\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:39.330479 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.330451    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-crio-socket\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:39.330479 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.330468    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a099b798-61cd-4306-85ba-85c7deb25a18-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-5lpq9\" (UID: \"a099b798-61cd-4306-85ba-85c7deb25a18\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-5lpq9"
Oct 30 21:37:39.330688 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.330489    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-etc-sysctl-conf\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:37:39.330688 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.330499    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6e615be3-41d7-4f82-90d7-7ce71ed9190b-dbus\") pod \"global-pull-secret-syncer-z9h2r\" (UID: \"6e615be3-41d7-4f82-90d7-7ce71ed9190b\") " pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:37:39.330688 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.330493    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-run-openvswitch\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.330688 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.330530    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-crio-socket\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:39.330688 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.330549    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6e615be3-41d7-4f82-90d7-7ce71ed9190b-kubelet-config\") pod \"global-pull-secret-syncer-z9h2r\" (UID: \"6e615be3-41d7-4f82-90d7-7ce71ed9190b\") " pod="kube-system/global-pull-secret-syncer-z9h2r"
Oct 30 21:37:39.330688 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.330578    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cb8bde01-2bc6-4301-aa2f-2858158bd54e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4frz9\" (UID: \"cb8bde01-2bc6-4301-aa2f-2858158bd54e\") " pod="openshift-multus/multus-additional-cni-plugins-4frz9"
Oct 30 21:37:39.330688 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.330619    4871 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 21:37:39.330688 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.330619    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cb8bde01-2bc6-4301-aa2f-2858158bd54e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4frz9\" (UID: \"cb8bde01-2bc6-4301-aa2f-2858158bd54e\") " pod="openshift-multus/multus-additional-cni-plugins-4frz9"
Oct 30 21:37:39.330688 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.330621    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e0535562-96d7-4fc4-a5d5-ad4cc785080e-service-ca-bundle podName:e0535562-96d7-4fc4-a5d5-ad4cc785080e nodeName:}" failed. No retries permitted until 2026-10-30 21:37:39.830596884 +0000 UTC m=+2.008085500 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e0535562-96d7-4fc4-a5d5-ad4cc785080e-service-ca-bundle") pod "router-default-57f669cfc9-d755h" (UID: "e0535562-96d7-4fc4-a5d5-ad4cc785080e") : configmap references non-existent config key: service-ca.crt
Oct 30 21:37:39.330688 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.330686    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-device-dir\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:37:39.330963 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.330705    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-run-systemd\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.330963 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.330709    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-etc-sysctl-conf\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:37:39.330963 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.330722    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-host-var-lib-cni-multus\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.330963 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.330723    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-device-dir\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:37:39.330963 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.330750    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a099b798-61cd-4306-85ba-85c7deb25a18-networking-console-plugin-cert podName:a099b798-61cd-4306-85ba-85c7deb25a18 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:39.830731482 +0000 UTC m=+2.008220086 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a099b798-61cd-4306-85ba-85c7deb25a18-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-5lpq9" (UID: "a099b798-61cd-4306-85ba-85c7deb25a18") : secret "networking-console-plugin-cert" not found
Oct 30 21:37:39.330963 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.330753    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-run-systemd\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.330963 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.330773    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-etc-sysctl-d\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:37:39.330963 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.330792    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-host-var-lib-cni-multus\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.330963 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.330807    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-multus-socket-dir-parent\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.330963 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.330828    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-run-ovn-kubernetes\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.330963 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.330845    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-etc-kubernetes\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.330963 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.330851    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-etc-sysctl-d\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:37:39.330963 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.330860    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-etc-systemd\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:37:39.330963 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.330903    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-run-ovn-kubernetes\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.330963 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.330910    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-etc-kubernetes\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.330963 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.330914    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-multus-socket-dir-parent\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.330963 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.330930    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-slash\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.330963 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.330947    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-etc-systemd\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:37:39.330963 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.330960    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cb8bde01-2bc6-4301-aa2f-2858158bd54e-cnibin\") pod \"multus-additional-cni-plugins-4frz9\" (UID: \"cb8bde01-2bc6-4301-aa2f-2858158bd54e\") " pod="openshift-multus/multus-additional-cni-plugins-4frz9"
Oct 30 21:37:39.330963 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.330971    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-slash\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.330981    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-host-run-k8s-cni-cncf-io\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.330997    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-run-netns\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331011    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-cni-netd\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331016    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cb8bde01-2bc6-4301-aa2f-2858158bd54e-cnibin\") pod \"multus-additional-cni-plugins-4frz9\" (UID: \"cb8bde01-2bc6-4301-aa2f-2858158bd54e\") " pod="openshift-multus/multus-additional-cni-plugins-4frz9"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331036    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331046    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-run-netns\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331051    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-host-run-k8s-cni-cncf-io\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331062    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-cni-netd\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331065    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb8bde01-2bc6-4301-aa2f-2858158bd54e-system-cni-dir\") pod \"multus-additional-cni-plugins-4frz9\" (UID: \"cb8bde01-2bc6-4301-aa2f-2858158bd54e\") " pod="openshift-multus/multus-additional-cni-plugins-4frz9"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331088    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331105    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-registry-tls\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331088    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331139    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-etc-modprobe-d\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331163    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-host-var-lib-cni-bin\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331166    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb8bde01-2bc6-4301-aa2f-2858158bd54e-system-cni-dir\") pod \"multus-additional-cni-plugins-4frz9\" (UID: \"cb8bde01-2bc6-4301-aa2f-2858158bd54e\") " pod="openshift-multus/multus-additional-cni-plugins-4frz9"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.331171    4871 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331177    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-hostroot\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.331190    4871 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.331203    4871 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55fbc4d8d7-w4mrq: secret "image-registry-tls" not found
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331210    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-host-run-multus-certs\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.331230    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-insights-runtime-extractor-tls podName:565e807a-a9c3-4f1b-8de0-9ff4d713a1ee nodeName:}" failed. No retries permitted until 2026-10-30 21:37:39.831214287 +0000 UTC m=+2.008702883 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-insights-runtime-extractor-tls") pod "insights-runtime-extractor-h6hzg" (UID: "565e807a-a9c3-4f1b-8de0-9ff4d713a1ee") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331265    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-host-var-lib-cni-bin\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331192    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-host-run-multus-certs\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331278    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-etc-modprobe-d\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331288    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-hostroot\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.331297    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-registry-tls podName:0dca73b7-6526-48ac-a76e-70e441debc00 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:39.831286839 +0000 UTC m=+2.008775455 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-registry-tls") pod "image-registry-55fbc4d8d7-w4mrq" (UID: "0dca73b7-6526-48ac-a76e-70e441debc00") : secret "image-registry-tls" not found
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331339    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-kubelet\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331371    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-systemd-units\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331395    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-etc-kubernetes\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331408    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-kubelet\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331417    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-systemd-units\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331421    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-multus-cni-dir\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331448    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-etc-kubernetes\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331449    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-etc-sysconfig\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331477    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5cf90653-8f33-4bb2-b622-1dae9144a9fe-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-q2pns\" (UID: \"5cf90653-8f33-4bb2-b622-1dae9144a9fe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q2pns"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331482    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-etc-sysconfig\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331481    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-multus-cni-dir\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331497    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/872bccf3-b52e-4c52-9f5e-0487b4957b50-metrics-tls\") pod \"dns-default-6q6pm\" (UID: \"872bccf3-b52e-4c52-9f5e-0487b4957b50\") " pod="openshift-dns/dns-default-6q6pm"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331517    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-multus-conf-dir\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331533    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/07b45ded-5ba8-49dc-b6ab-c170ca47af9b-host-slash\") pod \"iptables-alerter-2wb5p\" (UID: \"07b45ded-5ba8-49dc-b6ab-c170ca47af9b\") " pod="openshift-network-operator/iptables-alerter-2wb5p"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331549    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331565    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/909906d5-5049-4742-9b5a-b9c03ed5cc39-hosts-file\") pod \"node-resolver-dbpfd\" (UID: \"909906d5-5049-4742-9b5a-b9c03ed5cc39\") " pod="openshift-dns/node-resolver-dbpfd"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331595    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-multus-conf-dir\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.331585    4871 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331636    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331639    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-host-var-lib-kubelet\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331642    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/909906d5-5049-4742-9b5a-b9c03ed5cc39-hosts-file\") pod \"node-resolver-dbpfd\" (UID: \"909906d5-5049-4742-9b5a-b9c03ed5cc39\") " pod="openshift-dns/node-resolver-dbpfd"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.331640    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/872bccf3-b52e-4c52-9f5e-0487b4957b50-metrics-tls podName:872bccf3-b52e-4c52-9f5e-0487b4957b50 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:39.831629068 +0000 UTC m=+2.009117665 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/872bccf3-b52e-4c52-9f5e-0487b4957b50-metrics-tls") pod "dns-default-6q6pm" (UID: "872bccf3-b52e-4c52-9f5e-0487b4957b50") : secret "dns-default-metrics-tls" not found
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331605    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-host-var-lib-kubelet\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.331650    4871 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 21:37:39.331658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331681    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-etc-openvswitch\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331708    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-socket-dir\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331600    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/07b45ded-5ba8-49dc-b6ab-c170ca47af9b-host-slash\") pod \"iptables-alerter-2wb5p\" (UID: \"07b45ded-5ba8-49dc-b6ab-c170ca47af9b\") " pod="openshift-network-operator/iptables-alerter-2wb5p"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.331727    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cf90653-8f33-4bb2-b622-1dae9144a9fe-samples-operator-tls podName:5cf90653-8f33-4bb2-b622-1dae9144a9fe nodeName:}" failed. No retries permitted until 2026-10-30 21:37:39.831713146 +0000 UTC m=+2.009201759 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5cf90653-8f33-4bb2-b622-1dae9144a9fe-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-q2pns" (UID: "5cf90653-8f33-4bb2-b622-1dae9144a9fe") : secret "samples-operator-tls" not found
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331732    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-etc-openvswitch\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331746    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-socket-dir\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331764    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-host\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331780    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-sys-fs\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331796    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-host-run-netns\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331813    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/948affe7-aaf8-4d3e-a515-ed9552d19d90-host\") pod \"node-ca-grk5c\" (UID: \"948affe7-aaf8-4d3e-a515-ed9552d19d90\") " pod="openshift-image-registry/node-ca-grk5c"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331813    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-host\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331834    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-system-cni-dir\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331848    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-host-run-netns\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331848    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-sys-fs\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331853    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-log-socket\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331886    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-log-socket\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331890    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-system-cni-dir\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331856    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/948affe7-aaf8-4d3e-a515-ed9552d19d90-host\") pod \"node-ca-grk5c\" (UID: \"948affe7-aaf8-4d3e-a515-ed9552d19d90\") " pod="openshift-image-registry/node-ca-grk5c"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331901    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-sys\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331926    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-var-lib-openvswitch\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331942    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-var-lib-kubelet\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331957    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-node-log\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331969    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-var-lib-openvswitch\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331979    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-registration-dir\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331986    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-sys\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331993    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-node-log\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331992    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-var-lib-kubelet\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.332027    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-registration-dir\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.331997    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.332053    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-run-ovn\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.332032    4871 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.332075    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0535562-96d7-4fc4-a5d5-ad4cc785080e-metrics-certs\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.332103    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-run-ovn\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.332109    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/65827025-dc76-42d8-be9f-bdfed6b28429-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-wltlg\" (UID: \"65827025-dc76-42d8-be9f-bdfed6b28429\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wltlg"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.332119    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-kube-rbac-proxy-cm podName:565e807a-a9c3-4f1b-8de0-9ff4d713a1ee nodeName:}" failed. No retries permitted until 2026-10-30 21:37:39.832104435 +0000 UTC m=+2.009593046 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-kube-rbac-proxy-cm") pod "insights-runtime-extractor-h6hzg" (UID: "565e807a-a9c3-4f1b-8de0-9ff4d713a1ee") : configmap "kube-rbac-proxy" not found
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.332123    4871 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.332156    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0535562-96d7-4fc4-a5d5-ad4cc785080e-metrics-certs podName:e0535562-96d7-4fc4-a5d5-ad4cc785080e nodeName:}" failed. No retries permitted until 2026-10-30 21:37:39.832144871 +0000 UTC m=+2.009633478 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e0535562-96d7-4fc4-a5d5-ad4cc785080e-metrics-certs") pod "router-default-57f669cfc9-d755h" (UID: "e0535562-96d7-4fc4-a5d5-ad4cc785080e") : secret "router-metrics-certs-default" not found
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.332160    4871 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.332173    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-run\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.332195    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65827025-dc76-42d8-be9f-bdfed6b28429-cluster-monitoring-operator-tls podName:65827025-dc76-42d8-be9f-bdfed6b28429 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:39.832183552 +0000 UTC m=+2.009672159 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/65827025-dc76-42d8-be9f-bdfed6b28429-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-wltlg" (UID: "65827025-dc76-42d8-be9f-bdfed6b28429") : secret "cluster-monitoring-operator-tls" not found
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.332207    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-tls\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.332225    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-etc-selinux\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.332225    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-run\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.332276    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7b99dea5-32ac-4c1c-af68-b5d2b7af23c2-etc-selinux\") pod \"aws-ebs-csi-driver-node-7w7f9\" (UID: \"7b99dea5-32ac-4c1c-af68-b5d2b7af23c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w7f9"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.332297    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-metrics-certs\") pod \"network-metrics-daemon-l9kxl\" (UID: \"7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2\") " pod="openshift-multus/network-metrics-daemon-l9kxl"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.332319    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-cnibin\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.332341    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-lib-modules\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.332356    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-cni-bin\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.332324    4871 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.332366    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/726700d7-c252-479e-abdd-18c0573102d1-cnibin\") pod \"multus-ggqpf\" (UID: \"726700d7-c252-479e-abdd-18c0573102d1\") " pod="openshift-multus/multus-ggqpf"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.332372    4871 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d487d746-hf6pr: secret "image-registry-tls" not found
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.332358    4871 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.332410    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2360fddc-52a5-42a5-82ca-a05aff84d217-host-cni-bin\") pod \"ovnkube-node-wzntd\" (UID: \"2360fddc-52a5-42a5-82ca-a05aff84d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.332441    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-tls podName:30931e26-479e-4ddb-8ce3-9b2b52dc0e8e nodeName:}" failed. No retries permitted until 2026-10-30 21:37:39.832432846 +0000 UTC m=+2.009921441 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-tls") pod "image-registry-7d487d746-hf6pr" (UID: "30931e26-479e-4ddb-8ce3-9b2b52dc0e8e") : secret "image-registry-tls" not found
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.332458    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/457fd859-0880-43f8-b712-2fd90c9debb0-lib-modules\") pod \"tuned-f9gtx\" (UID: \"457fd859-0880-43f8-b712-2fd90c9debb0\") " pod="openshift-cluster-node-tuning-operator/tuned-f9gtx"
Oct 30 21:37:39.333297 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.332476    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-metrics-certs podName:7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:39.832464082 +0000 UTC m=+2.009952699 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-metrics-certs") pod "network-metrics-daemon-l9kxl" (UID: "7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2") : secret "metrics-daemon-secret" not found
Oct 30 21:37:39.358651 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.358627    4871 scope.go:122] "RemoveContainer" containerID="b357894139be464ca4c0abc3e8e4b114e23bf2f0cc6df8213aaa1e84a3a6e451"
Oct 30 21:37:39.835669 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.835573    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:39.835669 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.835618    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0535562-96d7-4fc4-a5d5-ad4cc785080e-metrics-certs\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:39.835669 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.835638    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/65827025-dc76-42d8-be9f-bdfed6b28429-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-wltlg\" (UID: \"65827025-dc76-42d8-be9f-bdfed6b28429\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wltlg"
Oct 30 21:37:39.835669 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.835665    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-tls\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:39.835984 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.835691    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-metrics-certs\") pod \"network-metrics-daemon-l9kxl\" (UID: \"7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2\") " pod="openshift-multus/network-metrics-daemon-l9kxl"
Oct 30 21:37:39.835984 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.835726    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0535562-96d7-4fc4-a5d5-ad4cc785080e-service-ca-bundle\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:39.835984 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.835746    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a099b798-61cd-4306-85ba-85c7deb25a18-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-5lpq9\" (UID: \"a099b798-61cd-4306-85ba-85c7deb25a18\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-5lpq9"
Oct 30 21:37:39.835984 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.835722    4871 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 21:37:39.835984 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.835782    4871 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 21:37:39.835984 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.835801    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:39.835984 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.835824    4871 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 21:37:39.835984 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.835845    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-metrics-certs podName:7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:40.835826808 +0000 UTC m=+3.013315423 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-metrics-certs") pod "network-metrics-daemon-l9kxl" (UID: "7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2") : secret "metrics-daemon-secret" not found
Oct 30 21:37:39.835984 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.835865    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-kube-rbac-proxy-cm podName:565e807a-a9c3-4f1b-8de0-9ff4d713a1ee nodeName:}" failed. No retries permitted until 2026-10-30 21:37:40.835855764 +0000 UTC m=+3.013344358 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-kube-rbac-proxy-cm") pod "insights-runtime-extractor-h6hzg" (UID: "565e807a-a9c3-4f1b-8de0-9ff4d713a1ee") : configmap "kube-rbac-proxy" not found
Oct 30 21:37:39.835984 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.835827    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-registry-tls\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:39.835984 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.835781    4871 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 21:37:39.835984 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.835880    4871 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 21:37:39.835984 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.835894    4871 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55fbc4d8d7-w4mrq: secret "image-registry-tls" not found
Oct 30 21:37:39.835984 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.835901    4871 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 21:37:39.835984 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.835791    4871 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 21:37:39.835984 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.835937    4871 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d487d746-hf6pr: secret "image-registry-tls" not found
Oct 30 21:37:39.835984 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.835880    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e0535562-96d7-4fc4-a5d5-ad4cc785080e-service-ca-bundle podName:e0535562-96d7-4fc4-a5d5-ad4cc785080e nodeName:}" failed. No retries permitted until 2026-10-30 21:37:40.835871616 +0000 UTC m=+3.013360210 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e0535562-96d7-4fc4-a5d5-ad4cc785080e-service-ca-bundle") pod "router-default-57f669cfc9-d755h" (UID: "e0535562-96d7-4fc4-a5d5-ad4cc785080e") : configmap references non-existent config key: service-ca.crt
Oct 30 21:37:39.835984 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.835955    4871 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:37:39.835984 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.835969    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a099b798-61cd-4306-85ba-85c7deb25a18-networking-console-plugin-cert podName:a099b798-61cd-4306-85ba-85c7deb25a18 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:40.835955185 +0000 UTC m=+3.013443785 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a099b798-61cd-4306-85ba-85c7deb25a18-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-5lpq9" (UID: "a099b798-61cd-4306-85ba-85c7deb25a18") : secret "networking-console-plugin-cert" not found
Oct 30 21:37:39.835984 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.835985    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0535562-96d7-4fc4-a5d5-ad4cc785080e-metrics-certs podName:e0535562-96d7-4fc4-a5d5-ad4cc785080e nodeName:}" failed. No retries permitted until 2026-10-30 21:37:40.835977869 +0000 UTC m=+3.013466464 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e0535562-96d7-4fc4-a5d5-ad4cc785080e-metrics-certs") pod "router-default-57f669cfc9-d755h" (UID: "e0535562-96d7-4fc4-a5d5-ad4cc785080e") : secret "router-metrics-certs-default" not found
Oct 30 21:37:39.835984 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.835997    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65827025-dc76-42d8-be9f-bdfed6b28429-cluster-monitoring-operator-tls podName:65827025-dc76-42d8-be9f-bdfed6b28429 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:40.835991385 +0000 UTC m=+3.013479980 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/65827025-dc76-42d8-be9f-bdfed6b28429-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-wltlg" (UID: "65827025-dc76-42d8-be9f-bdfed6b28429") : secret "cluster-monitoring-operator-tls" not found
Oct 30 21:37:39.836591 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.836007    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-registry-tls podName:0dca73b7-6526-48ac-a76e-70e441debc00 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:40.836002742 +0000 UTC m=+3.013491337 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-registry-tls") pod "image-registry-55fbc4d8d7-w4mrq" (UID: "0dca73b7-6526-48ac-a76e-70e441debc00") : secret "image-registry-tls" not found
Oct 30 21:37:39.836591 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.836019    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-tls podName:30931e26-479e-4ddb-8ce3-9b2b52dc0e8e nodeName:}" failed. No retries permitted until 2026-10-30 21:37:40.836015069 +0000 UTC m=+3.013503664 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-tls") pod "image-registry-7d487d746-hf6pr" (UID: "30931e26-479e-4ddb-8ce3-9b2b52dc0e8e") : secret "image-registry-tls" not found
Oct 30 21:37:39.836591 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.836035    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-insights-runtime-extractor-tls podName:565e807a-a9c3-4f1b-8de0-9ff4d713a1ee nodeName:}" failed. No retries permitted until 2026-10-30 21:37:40.836030684 +0000 UTC m=+3.013519279 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-insights-runtime-extractor-tls") pod "insights-runtime-extractor-h6hzg" (UID: "565e807a-a9c3-4f1b-8de0-9ff4d713a1ee") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:37:39.836591 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.836064    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5cf90653-8f33-4bb2-b622-1dae9144a9fe-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-q2pns\" (UID: \"5cf90653-8f33-4bb2-b622-1dae9144a9fe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q2pns"
Oct 30 21:37:39.836591 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:39.836083    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/872bccf3-b52e-4c52-9f5e-0487b4957b50-metrics-tls\") pod \"dns-default-6q6pm\" (UID: \"872bccf3-b52e-4c52-9f5e-0487b4957b50\") " pod="openshift-dns/dns-default-6q6pm"
Oct 30 21:37:39.836591 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.836144    4871 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:37:39.836591 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.836168    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/872bccf3-b52e-4c52-9f5e-0487b4957b50-metrics-tls podName:872bccf3-b52e-4c52-9f5e-0487b4957b50 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:40.836162627 +0000 UTC m=+3.013651222 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/872bccf3-b52e-4c52-9f5e-0487b4957b50-metrics-tls") pod "dns-default-6q6pm" (UID: "872bccf3-b52e-4c52-9f5e-0487b4957b50") : secret "dns-default-metrics-tls" not found
Oct 30 21:37:39.836591 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.836169    4871 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 21:37:39.836591 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:39.836196    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cf90653-8f33-4bb2-b622-1dae9144a9fe-samples-operator-tls podName:5cf90653-8f33-4bb2-b622-1dae9144a9fe nodeName:}" failed. No retries permitted until 2026-10-30 21:37:40.836189829 +0000 UTC m=+3.013678458 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5cf90653-8f33-4bb2-b622-1dae9144a9fe-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-q2pns" (UID: "5cf90653-8f33-4bb2-b622-1dae9144a9fe") : secret "samples-operator-tls" not found
Oct 30 21:37:40.011995 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:40.011970    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/2.log"
Oct 30 21:37:40.012425 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:40.012408    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/1.log"
Oct 30 21:37:40.012474 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:40.012438    4871 generic.go:356] "Generic (PLEG): container finished" podID="b7002d6d-834c-4bdc-907e-805edd577a40" containerID="901686daa2114aaf5caa204af594595f5e26c750babd8486812940305acf2f5d" exitCode=255
Oct 30 21:37:40.012537 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:40.012516    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f" event={"ID":"b7002d6d-834c-4bdc-907e-805edd577a40","Type":"ContainerDied","Data":"901686daa2114aaf5caa204af594595f5e26c750babd8486812940305acf2f5d"}
Oct 30 21:37:40.012586 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:40.012576    4871 scope.go:122] "RemoveContainer" containerID="b357894139be464ca4c0abc3e8e4b114e23bf2f0cc6df8213aaa1e84a3a6e451"
Oct 30 21:37:40.012863 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:40.012848    4871 scope.go:122] "RemoveContainer" containerID="901686daa2114aaf5caa204af594595f5e26c750babd8486812940305acf2f5d"
Oct 30 21:37:40.013056 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:40.013038    4871 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-7tf9f_openshift-console-operator(b7002d6d-834c-4bdc-907e-805edd577a40)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f" podUID="b7002d6d-834c-4bdc-907e-805edd577a40"
Oct 30 21:37:40.043488 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:40.043455    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:40.195444 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:40.195410    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:40.843788 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:40.843753    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0535562-96d7-4fc4-a5d5-ad4cc785080e-service-ca-bundle\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:40.843788 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:40.843789    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a099b798-61cd-4306-85ba-85c7deb25a18-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-5lpq9\" (UID: \"a099b798-61cd-4306-85ba-85c7deb25a18\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-5lpq9"
Oct 30 21:37:40.844103 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:40.843846    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:40.844103 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:40.843873    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-registry-tls\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:40.844103 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:40.843910    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5cf90653-8f33-4bb2-b622-1dae9144a9fe-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-q2pns\" (UID: \"5cf90653-8f33-4bb2-b622-1dae9144a9fe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q2pns"
Oct 30 21:37:40.844103 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:40.843941    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/872bccf3-b52e-4c52-9f5e-0487b4957b50-metrics-tls\") pod \"dns-default-6q6pm\" (UID: \"872bccf3-b52e-4c52-9f5e-0487b4957b50\") " pod="openshift-dns/dns-default-6q6pm"
Oct 30 21:37:40.844103 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:40.843982    4871 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 21:37:40.844103 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:40.843988    4871 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:37:40.844103 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:40.844007    4871 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:37:40.844103 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:40.843985    4871 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 21:37:40.844103 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:40.843988    4871 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 21:37:40.844103 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:40.843989    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e0535562-96d7-4fc4-a5d5-ad4cc785080e-service-ca-bundle podName:e0535562-96d7-4fc4-a5d5-ad4cc785080e nodeName:}" failed. No retries permitted until 2026-10-30 21:37:42.843958477 +0000 UTC m=+5.021447084 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e0535562-96d7-4fc4-a5d5-ad4cc785080e-service-ca-bundle") pod "router-default-57f669cfc9-d755h" (UID: "e0535562-96d7-4fc4-a5d5-ad4cc785080e") : configmap references non-existent config key: service-ca.crt
Oct 30 21:37:40.844103 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:40.843998    4871 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55fbc4d8d7-w4mrq: secret "image-registry-tls" not found
Oct 30 21:37:40.844103 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:40.844088    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-insights-runtime-extractor-tls podName:565e807a-a9c3-4f1b-8de0-9ff4d713a1ee nodeName:}" failed. No retries permitted until 2026-10-30 21:37:42.844076185 +0000 UTC m=+5.021564783 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-insights-runtime-extractor-tls") pod "insights-runtime-extractor-h6hzg" (UID: "565e807a-a9c3-4f1b-8de0-9ff4d713a1ee") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:37:40.844103 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:40.844109    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/872bccf3-b52e-4c52-9f5e-0487b4957b50-metrics-tls podName:872bccf3-b52e-4c52-9f5e-0487b4957b50 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:42.844099382 +0000 UTC m=+5.021587979 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/872bccf3-b52e-4c52-9f5e-0487b4957b50-metrics-tls") pod "dns-default-6q6pm" (UID: "872bccf3-b52e-4c52-9f5e-0487b4957b50") : secret "dns-default-metrics-tls" not found
Oct 30 21:37:40.844617 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:40.844133    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a099b798-61cd-4306-85ba-85c7deb25a18-networking-console-plugin-cert podName:a099b798-61cd-4306-85ba-85c7deb25a18 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:42.844124035 +0000 UTC m=+5.021612635 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a099b798-61cd-4306-85ba-85c7deb25a18-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-5lpq9" (UID: "a099b798-61cd-4306-85ba-85c7deb25a18") : secret "networking-console-plugin-cert" not found
Oct 30 21:37:40.844617 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:40.844155    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cf90653-8f33-4bb2-b622-1dae9144a9fe-samples-operator-tls podName:5cf90653-8f33-4bb2-b622-1dae9144a9fe nodeName:}" failed. No retries permitted until 2026-10-30 21:37:42.844146564 +0000 UTC m=+5.021635159 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5cf90653-8f33-4bb2-b622-1dae9144a9fe-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-q2pns" (UID: "5cf90653-8f33-4bb2-b622-1dae9144a9fe") : secret "samples-operator-tls" not found
Oct 30 21:37:40.844617 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:40.844176    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-registry-tls podName:0dca73b7-6526-48ac-a76e-70e441debc00 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:42.844167837 +0000 UTC m=+5.021656432 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-registry-tls") pod "image-registry-55fbc4d8d7-w4mrq" (UID: "0dca73b7-6526-48ac-a76e-70e441debc00") : secret "image-registry-tls" not found
Oct 30 21:37:40.844617 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:40.844230    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:40.844617 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:40.844270    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0535562-96d7-4fc4-a5d5-ad4cc785080e-metrics-certs\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:40.844617 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:40.844296    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/65827025-dc76-42d8-be9f-bdfed6b28429-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-wltlg\" (UID: \"65827025-dc76-42d8-be9f-bdfed6b28429\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wltlg"
Oct 30 21:37:40.844617 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:40.844324    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-tls\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:40.844617 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:40.844331    4871 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 21:37:40.844617 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:40.844339    4871 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 21:37:40.844617 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:40.844353    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-metrics-certs\") pod \"network-metrics-daemon-l9kxl\" (UID: \"7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2\") " pod="openshift-multus/network-metrics-daemon-l9kxl"
Oct 30 21:37:40.844617 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:40.844362    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0535562-96d7-4fc4-a5d5-ad4cc785080e-metrics-certs podName:e0535562-96d7-4fc4-a5d5-ad4cc785080e nodeName:}" failed. No retries permitted until 2026-10-30 21:37:42.844352471 +0000 UTC m=+5.021841093 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e0535562-96d7-4fc4-a5d5-ad4cc785080e-metrics-certs") pod "router-default-57f669cfc9-d755h" (UID: "e0535562-96d7-4fc4-a5d5-ad4cc785080e") : secret "router-metrics-certs-default" not found
Oct 30 21:37:40.844617 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:40.844376    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-kube-rbac-proxy-cm podName:565e807a-a9c3-4f1b-8de0-9ff4d713a1ee nodeName:}" failed. No retries permitted until 2026-10-30 21:37:42.844369927 +0000 UTC m=+5.021858522 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-kube-rbac-proxy-cm") pod "insights-runtime-extractor-h6hzg" (UID: "565e807a-a9c3-4f1b-8de0-9ff4d713a1ee") : configmap "kube-rbac-proxy" not found
Oct 30 21:37:40.844617 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:40.844406    4871 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 21:37:40.844617 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:40.844437    4871 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 21:37:40.844617 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:40.844440    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65827025-dc76-42d8-be9f-bdfed6b28429-cluster-monitoring-operator-tls podName:65827025-dc76-42d8-be9f-bdfed6b28429 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:42.844428865 +0000 UTC m=+5.021917462 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/65827025-dc76-42d8-be9f-bdfed6b28429-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-wltlg" (UID: "65827025-dc76-42d8-be9f-bdfed6b28429") : secret "cluster-monitoring-operator-tls" not found
Oct 30 21:37:40.844617 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:40.844462    4871 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 21:37:40.844617 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:40.844472    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-metrics-certs podName:7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:42.844465085 +0000 UTC m=+5.021953680 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-metrics-certs") pod "network-metrics-daemon-l9kxl" (UID: "7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2") : secret "metrics-daemon-secret" not found
Oct 30 21:37:40.844617 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:40.844479    4871 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d487d746-hf6pr: secret "image-registry-tls" not found
Oct 30 21:37:40.844617 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:40.844524    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-tls podName:30931e26-479e-4ddb-8ce3-9b2b52dc0e8e nodeName:}" failed. No retries permitted until 2026-10-30 21:37:42.844510188 +0000 UTC m=+5.021998790 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-tls") pod "image-registry-7d487d746-hf6pr" (UID: "30931e26-479e-4ddb-8ce3-9b2b52dc0e8e") : secret "image-registry-tls" not found
Oct 30 21:37:41.017341 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:41.017313    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/2.log"
Oct 30 21:37:41.055845 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:41.055813    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:41.315637 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:41.315598    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:41.412592 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:41.412567    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dbpfd_909906d5-5049-4742-9b5a-b9c03ed5cc39/dns-node-resolver/0.log"
Oct 30 21:37:41.677918 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:41.677881    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:41.918365 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:41.918332    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:42.174655 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:42.174628    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:42.402988 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:42.402953    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:42.612820 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:42.612795    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-grk5c_948affe7-aaf8-4d3e-a515-ed9552d19d90/node-ca/0.log"
Oct 30 21:37:42.736515 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:42.736483    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:42.862290 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:42.862231    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a099b798-61cd-4306-85ba-85c7deb25a18-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-5lpq9\" (UID: \"a099b798-61cd-4306-85ba-85c7deb25a18\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-5lpq9"
Oct 30 21:37:42.862444 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:42.862318    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:42.862444 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:42.862343    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-registry-tls\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:42.862444 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:42.862369    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5cf90653-8f33-4bb2-b622-1dae9144a9fe-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-q2pns\" (UID: \"5cf90653-8f33-4bb2-b622-1dae9144a9fe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q2pns"
Oct 30 21:37:42.862444 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:42.862382    4871 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 21:37:42.862444 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:42.862391    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/872bccf3-b52e-4c52-9f5e-0487b4957b50-metrics-tls\") pod \"dns-default-6q6pm\" (UID: \"872bccf3-b52e-4c52-9f5e-0487b4957b50\") " pod="openshift-dns/dns-default-6q6pm"
Oct 30 21:37:42.862677 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:42.862454    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a099b798-61cd-4306-85ba-85c7deb25a18-networking-console-plugin-cert podName:a099b798-61cd-4306-85ba-85c7deb25a18 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:46.862437929 +0000 UTC m=+9.039926538 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a099b798-61cd-4306-85ba-85c7deb25a18-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-5lpq9" (UID: "a099b798-61cd-4306-85ba-85c7deb25a18") : secret "networking-console-plugin-cert" not found
Oct 30 21:37:42.862677 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:42.862459    4871 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:37:42.862677 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:42.862481    4871 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 21:37:42.862677 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:42.862495    4871 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55fbc4d8d7-w4mrq: secret "image-registry-tls" not found
Oct 30 21:37:42.862677 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:42.862514    4871 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 21:37:42.862677 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:42.862519    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-insights-runtime-extractor-tls podName:565e807a-a9c3-4f1b-8de0-9ff4d713a1ee nodeName:}" failed. No retries permitted until 2026-10-30 21:37:46.86250094 +0000 UTC m=+9.039989552 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-insights-runtime-extractor-tls") pod "insights-runtime-extractor-h6hzg" (UID: "565e807a-a9c3-4f1b-8de0-9ff4d713a1ee") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:37:42.862677 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:42.862482    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:42.862677 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:42.862541    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-registry-tls podName:0dca73b7-6526-48ac-a76e-70e441debc00 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:46.862530831 +0000 UTC m=+9.040019426 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-registry-tls") pod "image-registry-55fbc4d8d7-w4mrq" (UID: "0dca73b7-6526-48ac-a76e-70e441debc00") : secret "image-registry-tls" not found
Oct 30 21:37:42.862677 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:42.862562    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-kube-rbac-proxy-cm podName:565e807a-a9c3-4f1b-8de0-9ff4d713a1ee nodeName:}" failed. No retries permitted until 2026-10-30 21:37:46.86255376 +0000 UTC m=+9.040042361 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-kube-rbac-proxy-cm") pod "insights-runtime-extractor-h6hzg" (UID: "565e807a-a9c3-4f1b-8de0-9ff4d713a1ee") : configmap "kube-rbac-proxy" not found
Oct 30 21:37:42.862677 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:42.862578    4871 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 21:37:42.862677 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:42.862606    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0535562-96d7-4fc4-a5d5-ad4cc785080e-metrics-certs\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:42.862677 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:42.862617    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cf90653-8f33-4bb2-b622-1dae9144a9fe-samples-operator-tls podName:5cf90653-8f33-4bb2-b622-1dae9144a9fe nodeName:}" failed. No retries permitted until 2026-10-30 21:37:46.862605536 +0000 UTC m=+9.040094160 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5cf90653-8f33-4bb2-b622-1dae9144a9fe-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-q2pns" (UID: "5cf90653-8f33-4bb2-b622-1dae9144a9fe") : secret "samples-operator-tls" not found
Oct 30 21:37:42.863288 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:42.862702    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/65827025-dc76-42d8-be9f-bdfed6b28429-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-wltlg\" (UID: \"65827025-dc76-42d8-be9f-bdfed6b28429\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wltlg"
Oct 30 21:37:42.863288 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:42.862744    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-tls\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:42.863288 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:42.862774    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-metrics-certs\") pod \"network-metrics-daemon-l9kxl\" (UID: \"7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2\") " pod="openshift-multus/network-metrics-daemon-l9kxl"
Oct 30 21:37:42.863288 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:42.862798    4871 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 21:37:42.863288 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:42.862821    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0535562-96d7-4fc4-a5d5-ad4cc785080e-service-ca-bundle\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:42.863288 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:42.862833    4871 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 21:37:42.863288 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:42.862849    4871 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d487d746-hf6pr: secret "image-registry-tls" not found
Oct 30 21:37:42.863288 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:42.862886    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-tls podName:30931e26-479e-4ddb-8ce3-9b2b52dc0e8e nodeName:}" failed. No retries permitted until 2026-10-30 21:37:46.862874557 +0000 UTC m=+9.040363167 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-tls") pod "image-registry-7d487d746-hf6pr" (UID: "30931e26-479e-4ddb-8ce3-9b2b52dc0e8e") : secret "image-registry-tls" not found
Oct 30 21:37:42.863288 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:42.862903    4871 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 21:37:42.863288 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:42.862941    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e0535562-96d7-4fc4-a5d5-ad4cc785080e-service-ca-bundle podName:e0535562-96d7-4fc4-a5d5-ad4cc785080e nodeName:}" failed. No retries permitted until 2026-10-30 21:37:46.862927447 +0000 UTC m=+9.040416061 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e0535562-96d7-4fc4-a5d5-ad4cc785080e-service-ca-bundle") pod "router-default-57f669cfc9-d755h" (UID: "e0535562-96d7-4fc4-a5d5-ad4cc785080e") : configmap references non-existent config key: service-ca.crt
Oct 30 21:37:42.863288 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:42.862962    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-metrics-certs podName:7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:46.862952288 +0000 UTC m=+9.040440886 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-metrics-certs") pod "network-metrics-daemon-l9kxl" (UID: "7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2") : secret "metrics-daemon-secret" not found
Oct 30 21:37:42.865095 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:42.865074    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/872bccf3-b52e-4c52-9f5e-0487b4957b50-metrics-tls\") pod \"dns-default-6q6pm\" (UID: \"872bccf3-b52e-4c52-9f5e-0487b4957b50\") " pod="openshift-dns/dns-default-6q6pm"
Oct 30 21:37:42.865185 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:42.865072    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/65827025-dc76-42d8-be9f-bdfed6b28429-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-wltlg\" (UID: \"65827025-dc76-42d8-be9f-bdfed6b28429\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wltlg"
Oct 30 21:37:42.865222 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:42.865181    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0535562-96d7-4fc4-a5d5-ad4cc785080e-metrics-certs\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:42.958042 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:42.958007    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wltlg"
Oct 30 21:37:42.969383 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:42.969353    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6q6pm"
Oct 30 21:37:43.089436 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:43.089407    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:43.089705 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:43.089689    4871 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 21:37:43.157193 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:43.157108    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:43.324795 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:43.324760    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-wpc6f"
Oct 30 21:37:43.372122 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:43.372090    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-wltlg"]
Oct 30 21:37:43.372558 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:43.372537    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:43.375706 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:43.375683    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65827025_dc76_42d8_be9f_bdfed6b28429.slice/crio-3e8db932306ce6089daf393dd0e81971f300a6fcb2e3ba55ea8ad27ff837e221 WatchSource:0}: Error finding container 3e8db932306ce6089daf393dd0e81971f300a6fcb2e3ba55ea8ad27ff837e221: Status 404 returned error can't find the container with id 3e8db932306ce6089daf393dd0e81971f300a6fcb2e3ba55ea8ad27ff837e221
Oct 30 21:37:43.377500 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:43.377476    4871 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:37:43.388893 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:43.388871    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6q6pm"]
Oct 30 21:37:43.392313 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:43.392292    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod872bccf3_b52e_4c52_9f5e_0487b4957b50.slice/crio-193a3b02cabfc4c913012983b4b527d1bd306379a96a2a1a95d7094182d635c7 WatchSource:0}: Error finding container 193a3b02cabfc4c913012983b4b527d1bd306379a96a2a1a95d7094182d635c7: Status 404 returned error can't find the container with id 193a3b02cabfc4c913012983b4b527d1bd306379a96a2a1a95d7094182d635c7
Oct 30 21:37:43.413185 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:43.413096    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rqxfc_be4fa9bf-41e0-4b26-9457-f9815a36d5b5/serve-healthcheck-canary/0.log"
Oct 30 21:37:43.566932 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:43.566865    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:43.947138 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:43.947105    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:44.017676 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:44.017651    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7d4bc45dd6-jgspb_37186e06-8d56-4c61-ba10-e0408d2e2fa4/kube-storage-version-migrator-operator/0.log"
Oct 30 21:37:44.029135 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:44.029103    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wltlg" event={"ID":"65827025-dc76-42d8-be9f-bdfed6b28429","Type":"ContainerStarted","Data":"3e8db932306ce6089daf393dd0e81971f300a6fcb2e3ba55ea8ad27ff837e221"}
Oct 30 21:37:44.030088 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:44.030057    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6q6pm" event={"ID":"872bccf3-b52e-4c52-9f5e-0487b4957b50","Type":"ContainerStarted","Data":"193a3b02cabfc4c913012983b4b527d1bd306379a96a2a1a95d7094182d635c7"}
Oct 30 21:37:44.330521 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:44.330438    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:44.730966 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:44.730929    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:45.132228 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:45.132185    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:45.533873 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:45.533797    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:45.744570 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:45.744539    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f"
Oct 30 21:37:45.745012 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:45.744994    4871 scope.go:122] "RemoveContainer" containerID="901686daa2114aaf5caa204af594595f5e26c750babd8486812940305acf2f5d"
Oct 30 21:37:45.745294 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:45.745231    4871 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-7tf9f_openshift-console-operator(b7002d6d-834c-4bdc-907e-805edd577a40)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f" podUID="b7002d6d-834c-4bdc-907e-805edd577a40"
Oct 30 21:37:45.932389 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:45.932332    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:45.957791 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:45.957764    4871 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f"
Oct 30 21:37:46.039835 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:46.039746    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6q6pm" event={"ID":"872bccf3-b52e-4c52-9f5e-0487b4957b50","Type":"ContainerStarted","Data":"05c986304a6d9f1cd1b28183ca0ab41455089b0b789cbea28810512235b1f1ec"}
Oct 30 21:37:46.039835 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:46.039805    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6q6pm" event={"ID":"872bccf3-b52e-4c52-9f5e-0487b4957b50","Type":"ContainerStarted","Data":"7fb23877ef1e262b2f6f6758e4b46076c2649c47166fc927c0c444f2edc74330"}
Oct 30 21:37:46.040058 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:46.040020    4871 scope.go:122] "RemoveContainer" containerID="901686daa2114aaf5caa204af594595f5e26c750babd8486812940305acf2f5d"
Oct 30 21:37:46.040249 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:46.040215    4871 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-7tf9f_openshift-console-operator(b7002d6d-834c-4bdc-907e-805edd577a40)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f" podUID="b7002d6d-834c-4bdc-907e-805edd577a40"
Oct 30 21:37:46.347049 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:46.346967    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:46.740353 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:46.740316    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:46.900743 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:46.900698    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0535562-96d7-4fc4-a5d5-ad4cc785080e-service-ca-bundle\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:46.900743 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:46.900746    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a099b798-61cd-4306-85ba-85c7deb25a18-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-5lpq9\" (UID: \"a099b798-61cd-4306-85ba-85c7deb25a18\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-5lpq9"
Oct 30 21:37:46.900979 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:46.900784    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:46.900979 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:46.900801    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-registry-tls\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:46.900979 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:46.900823    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5cf90653-8f33-4bb2-b622-1dae9144a9fe-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-q2pns\" (UID: \"5cf90653-8f33-4bb2-b622-1dae9144a9fe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q2pns"
Oct 30 21:37:46.900979 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:46.900873    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e0535562-96d7-4fc4-a5d5-ad4cc785080e-service-ca-bundle podName:e0535562-96d7-4fc4-a5d5-ad4cc785080e nodeName:}" failed. No retries permitted until 2026-10-30 21:37:54.900852108 +0000 UTC m=+17.078340704 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e0535562-96d7-4fc4-a5d5-ad4cc785080e-service-ca-bundle") pod "router-default-57f669cfc9-d755h" (UID: "e0535562-96d7-4fc4-a5d5-ad4cc785080e") : configmap references non-existent config key: service-ca.crt
Oct 30 21:37:46.900979 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:46.900902    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:46.900979 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:46.900929    4871 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 21:37:46.900979 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:46.900945    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-tls\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:46.900979 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:46.900966    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-metrics-certs\") pod \"network-metrics-daemon-l9kxl\" (UID: \"7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2\") " pod="openshift-multus/network-metrics-daemon-l9kxl"
Oct 30 21:37:46.900979 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:46.900967    4871 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 21:37:46.901396 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:46.900930    4871 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:37:46.901396 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:46.901011    4871 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 21:37:46.901396 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:46.901038    4871 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 21:37:46.901396 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:46.901052    4871 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d487d746-hf6pr: secret "image-registry-tls" not found
Oct 30 21:37:46.901396 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:46.900929    4871 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 21:37:46.901396 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:46.900936    4871 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 21:37:46.901396 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:46.901115    4871 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55fbc4d8d7-w4mrq: secret "image-registry-tls" not found
Oct 30 21:37:46.901396 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:46.900990    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a099b798-61cd-4306-85ba-85c7deb25a18-networking-console-plugin-cert podName:a099b798-61cd-4306-85ba-85c7deb25a18 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:54.900970791 +0000 UTC m=+17.078459400 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a099b798-61cd-4306-85ba-85c7deb25a18-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-5lpq9" (UID: "a099b798-61cd-4306-85ba-85c7deb25a18") : secret "networking-console-plugin-cert" not found
Oct 30 21:37:46.901396 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:46.901154    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-kube-rbac-proxy-cm podName:565e807a-a9c3-4f1b-8de0-9ff4d713a1ee nodeName:}" failed. No retries permitted until 2026-10-30 21:37:54.901144347 +0000 UTC m=+17.078632942 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-kube-rbac-proxy-cm") pod "insights-runtime-extractor-h6hzg" (UID: "565e807a-a9c3-4f1b-8de0-9ff4d713a1ee") : configmap "kube-rbac-proxy" not found
Oct 30 21:37:46.901396 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:46.901166    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-insights-runtime-extractor-tls podName:565e807a-a9c3-4f1b-8de0-9ff4d713a1ee nodeName:}" failed. No retries permitted until 2026-10-30 21:37:54.901160558 +0000 UTC m=+17.078649152 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-insights-runtime-extractor-tls") pod "insights-runtime-extractor-h6hzg" (UID: "565e807a-a9c3-4f1b-8de0-9ff4d713a1ee") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:37:46.901396 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:46.901177    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-metrics-certs podName:7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:54.901172415 +0000 UTC m=+17.078661009 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-metrics-certs") pod "network-metrics-daemon-l9kxl" (UID: "7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2") : secret "metrics-daemon-secret" not found
Oct 30 21:37:46.901396 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:46.901198    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-tls podName:30931e26-479e-4ddb-8ce3-9b2b52dc0e8e nodeName:}" failed. No retries permitted until 2026-10-30 21:37:54.901186735 +0000 UTC m=+17.078675336 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-tls") pod "image-registry-7d487d746-hf6pr" (UID: "30931e26-479e-4ddb-8ce3-9b2b52dc0e8e") : secret "image-registry-tls" not found
Oct 30 21:37:46.901396 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:46.901218    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cf90653-8f33-4bb2-b622-1dae9144a9fe-samples-operator-tls podName:5cf90653-8f33-4bb2-b622-1dae9144a9fe nodeName:}" failed. No retries permitted until 2026-10-30 21:37:54.901211972 +0000 UTC m=+17.078700566 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5cf90653-8f33-4bb2-b622-1dae9144a9fe-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-q2pns" (UID: "5cf90653-8f33-4bb2-b622-1dae9144a9fe") : secret "samples-operator-tls" not found
Oct 30 21:37:46.901396 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:37:46.901229    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-registry-tls podName:0dca73b7-6526-48ac-a76e-70e441debc00 nodeName:}" failed. No retries permitted until 2026-10-30 21:37:54.901224302 +0000 UTC m=+17.078712896 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-registry-tls") pod "image-registry-55fbc4d8d7-w4mrq" (UID: "0dca73b7-6526-48ac-a76e-70e441debc00") : secret "image-registry-tls" not found
Oct 30 21:37:47.045079 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:47.044979    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-wltlg" event={"ID":"65827025-dc76-42d8-be9f-bdfed6b28429","Type":"ContainerStarted","Data":"e393cc764a1b2599e9da003a60cc367e6fdb4a389a9c99daff43ac241709a6e5"}
Oct 30 21:37:47.045377 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:47.045354    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-6q6pm"
Oct 30 21:37:47.133574 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:47.133532    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:47.540209 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:47.540172    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:47.754724 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:47.754681    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:47.906179 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:47.906141    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:48.073792 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:48.073753    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:48.316398 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:48.316317    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:48.366475 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:48.366447    4871 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-46.us-west-2.compute.internal" event="NodeReady"
Oct 30 21:37:48.475285 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:48.475231    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:48.871247 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:48.871206    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:49.036423 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:49.036393    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:49.185689 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:49.185617    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:49.534675 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:49.534597    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:49.730307 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:49.730273    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:49.740164 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:49.740140    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wzntd"
Oct 30 21:37:49.953225 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:49.953195    4871 scope.go:122] "RemoveContainer" containerID="901686daa2114aaf5caa204af594595f5e26c750babd8486812940305acf2f5d"
Oct 30 21:37:50.057516 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:50.057491    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/2.log"
Oct 30 21:37:50.057673 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:50.057543    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f" event={"ID":"b7002d6d-834c-4bdc-907e-805edd577a40","Type":"ContainerStarted","Data":"0096f963ee14b5a89bfc7947da6fa1360f7f2f730eb41ad7556c7a7005773eae"}
Oct 30 21:37:50.057874 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:50.057852    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f"
Oct 30 21:37:50.058420 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:50.058395    4871 patch_prober.go:29] interesting pod/console-operator-7cd56b8dbc-7tf9f container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.133.0.17:8443/readyz\": dial tcp 10.133.0.17:8443: connect: connection refused" start-of-body=
Oct 30 21:37:50.058504 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:50.058453    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f" podUID="b7002d6d-834c-4bdc-907e-805edd577a40" containerName="console-operator" probeResult="failure" output="Get \"https://10.133.0.17:8443/readyz\": dial tcp 10.133.0.17:8443: connect: connection refused"
Oct 30 21:37:50.158786 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:50.158748    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:50.932150 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:50.932119    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:51.065442 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:51.065414    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-7tf9f"
Oct 30 21:37:51.302287 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:51.302201    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-4vb42"
Oct 30 21:37:51.364542 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:51.364509    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:51.552746 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:51.552663    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:51.709121 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:51.709090    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:51.868977 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:51.868945    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:52.024544 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:52.024510    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:52.232381 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:52.232299    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:52.532184 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:52.532100    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:52.729470 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:52.729439    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:54.685652 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:54.685617    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:54.844197 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:54.844168    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:54.977268 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:54.977171    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:54.977268 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:54.977212    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-tls\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:54.977268 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:54.977230    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-metrics-certs\") pod \"network-metrics-daemon-l9kxl\" (UID: \"7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2\") " pod="openshift-multus/network-metrics-daemon-l9kxl"
Oct 30 21:37:54.977535 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:54.977275    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0535562-96d7-4fc4-a5d5-ad4cc785080e-service-ca-bundle\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:54.977535 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:54.977294    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a099b798-61cd-4306-85ba-85c7deb25a18-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-5lpq9\" (UID: \"a099b798-61cd-4306-85ba-85c7deb25a18\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-5lpq9"
Oct 30 21:37:54.977535 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:54.977457    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:54.977535 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:54.977475    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-registry-tls\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:54.977535 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:54.977504    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5cf90653-8f33-4bb2-b622-1dae9144a9fe-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-q2pns\" (UID: \"5cf90653-8f33-4bb2-b622-1dae9144a9fe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q2pns"
Oct 30 21:37:54.977994 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:54.977968    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:54.978136 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:54.978055    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0535562-96d7-4fc4-a5d5-ad4cc785080e-service-ca-bundle\") pod \"router-default-57f669cfc9-d755h\" (UID: \"e0535562-96d7-4fc4-a5d5-ad4cc785080e\") " pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:54.980031 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:54.980008    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a099b798-61cd-4306-85ba-85c7deb25a18-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-5lpq9\" (UID: \"a099b798-61cd-4306-85ba-85c7deb25a18\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-5lpq9"
Oct 30 21:37:54.980115 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:54.980034    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2-metrics-certs\") pod \"network-metrics-daemon-l9kxl\" (UID: \"7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2\") " pod="openshift-multus/network-metrics-daemon-l9kxl"
Oct 30 21:37:54.980306 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:54.980289    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-tls\") pod \"image-registry-7d487d746-hf6pr\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") " pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:54.980393 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:54.980372    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/565e807a-a9c3-4f1b-8de0-9ff4d713a1ee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-h6hzg\" (UID: \"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee\") " pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:54.980458 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:54.980436    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5cf90653-8f33-4bb2-b622-1dae9144a9fe-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-q2pns\" (UID: \"5cf90653-8f33-4bb2-b622-1dae9144a9fe\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q2pns"
Oct 30 21:37:54.980550 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:54.980535    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-registry-tls\") pod \"image-registry-55fbc4d8d7-w4mrq\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") " pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:55.257485 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:55.257390    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q2pns"
Oct 30 21:37:55.259558 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:55.259537    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:55.260152 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:55.259828    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:55.260603 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:55.260582    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-5lpq9"
Oct 30 21:37:55.264007 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:55.263988    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9kxl"
Oct 30 21:37:55.264095 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:55.264034    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:55.270886 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:55.270868    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-h6hzg"
Oct 30 21:37:55.709293 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:55.709263    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-5lpq9"]
Oct 30 21:37:55.714215 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:55.714184    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda099b798_61cd_4306_85ba_85c7deb25a18.slice/crio-e100e5bbeb76ffd582eac023814e6d90ef4e73fb124257680e5c20e8caca2999 WatchSource:0}: Error finding container e100e5bbeb76ffd582eac023814e6d90ef4e73fb124257680e5c20e8caca2999: Status 404 returned error can't find the container with id e100e5bbeb76ffd582eac023814e6d90ef4e73fb124257680e5c20e8caca2999
Oct 30 21:37:55.755993 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:55.755970    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q2pns"]
Oct 30 21:37:55.758764 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:55.758724    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-h6hzg"]
Oct 30 21:37:55.764399 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:55.764109    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod565e807a_a9c3_4f1b_8de0_9ff4d713a1ee.slice/crio-4012ed9bceb0fa4cfb7c8d00dc84a78191b56779e5e1f7c0aba2a099d76191a7 WatchSource:0}: Error finding container 4012ed9bceb0fa4cfb7c8d00dc84a78191b56779e5e1f7c0aba2a099d76191a7: Status 404 returned error can't find the container with id 4012ed9bceb0fa4cfb7c8d00dc84a78191b56779e5e1f7c0aba2a099d76191a7
Oct 30 21:37:55.769347 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:55.769310    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0535562_96d7_4fc4_a5d5_ad4cc785080e.slice/crio-55213d48a8222a35ff2d84666989160177b2d3f1bdb638c66d4d0e0b04f3203a WatchSource:0}: Error finding container 55213d48a8222a35ff2d84666989160177b2d3f1bdb638c66d4d0e0b04f3203a: Status 404 returned error can't find the container with id 55213d48a8222a35ff2d84666989160177b2d3f1bdb638c66d4d0e0b04f3203a
Oct 30 21:37:55.769571 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:55.769547    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dca73b7_6526_48ac_a76e_70e441debc00.slice/crio-cd1aab9d007f228e04db2b8ee9e607316cb2624c20cab7fc88dc3a1d2d1a7d25 WatchSource:0}: Error finding container cd1aab9d007f228e04db2b8ee9e607316cb2624c20cab7fc88dc3a1d2d1a7d25: Status 404 returned error can't find the container with id cd1aab9d007f228e04db2b8ee9e607316cb2624c20cab7fc88dc3a1d2d1a7d25
Oct 30 21:37:55.803445 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:55.803365    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-57f669cfc9-d755h"]
Oct 30 21:37:55.807452 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:55.805815    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"]
Oct 30 21:37:55.807452 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:55.806773    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b0aa12e_a4c2_4f25_96f0_5c2272f8caf2.slice/crio-7d0eabcbac53f615c5bd3ba585557b229e0e9bcbff6d2330f697e529d51cc605 WatchSource:0}: Error finding container 7d0eabcbac53f615c5bd3ba585557b229e0e9bcbff6d2330f697e529d51cc605: Status 404 returned error can't find the container with id 7d0eabcbac53f615c5bd3ba585557b229e0e9bcbff6d2330f697e529d51cc605
Oct 30 21:37:55.807452 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:55.807072    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-l9kxl"]
Oct 30 21:37:55.814896 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:37:55.814864    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30931e26_479e_4ddb_8ce3_9b2b52dc0e8e.slice/crio-574c8ccb000cf04aae3ef1c987748b7ad799355055cff9bf263ec7f309bb5107 WatchSource:0}: Error finding container 574c8ccb000cf04aae3ef1c987748b7ad799355055cff9bf263ec7f309bb5107: Status 404 returned error can't find the container with id 574c8ccb000cf04aae3ef1c987748b7ad799355055cff9bf263ec7f309bb5107
Oct 30 21:37:55.849475 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:55.849446    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d487d746-hf6pr"]
Oct 30 21:37:56.084174 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:56.084077    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-h6hzg" event={"ID":"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee","Type":"ContainerStarted","Data":"f750af04184721ecdf8f84fe85788462e3f41e5e46ca2dd0b81a9377dfea3ec5"}
Oct 30 21:37:56.084340 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:56.084210    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-h6hzg" event={"ID":"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee","Type":"ContainerStarted","Data":"4012ed9bceb0fa4cfb7c8d00dc84a78191b56779e5e1f7c0aba2a099d76191a7"}
Oct 30 21:37:56.089970 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:56.089939    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-5lpq9" event={"ID":"a099b798-61cd-4306-85ba-85c7deb25a18","Type":"ContainerStarted","Data":"e100e5bbeb76ffd582eac023814e6d90ef4e73fb124257680e5c20e8caca2999"}
Oct 30 21:37:56.091781 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:56.091756    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-57f669cfc9-d755h" event={"ID":"e0535562-96d7-4fc4-a5d5-ad4cc785080e","Type":"ContainerStarted","Data":"24f49fcc2fece447510020e32c02fdbec7b395fe07aa6d628d5f0f3b1afdd3e2"}
Oct 30 21:37:56.091903 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:56.091797    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-57f669cfc9-d755h" event={"ID":"e0535562-96d7-4fc4-a5d5-ad4cc785080e","Type":"ContainerStarted","Data":"55213d48a8222a35ff2d84666989160177b2d3f1bdb638c66d4d0e0b04f3203a"}
Oct 30 21:37:56.093332 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:56.093305    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq" event={"ID":"0dca73b7-6526-48ac-a76e-70e441debc00","Type":"ContainerStarted","Data":"a5c9c0b4ad24703407193b74a120de5f0cb8bdedfc45aae069e612b364c1171e"}
Oct 30 21:37:56.093457 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:56.093347    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq" event={"ID":"0dca73b7-6526-48ac-a76e-70e441debc00","Type":"ContainerStarted","Data":"cd1aab9d007f228e04db2b8ee9e607316cb2624c20cab7fc88dc3a1d2d1a7d25"}
Oct 30 21:37:56.093457 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:56.093367    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:37:56.094811 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:56.094787    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l9kxl" event={"ID":"7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2","Type":"ContainerStarted","Data":"7d0eabcbac53f615c5bd3ba585557b229e0e9bcbff6d2330f697e529d51cc605"}
Oct 30 21:37:56.095994 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:56.095971    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q2pns" event={"ID":"5cf90653-8f33-4bb2-b622-1dae9144a9fe","Type":"ContainerStarted","Data":"9ce71071ff913d7caeda9d31442e32fcc1c6d4bc8e3fb64fafa4b25b7c823548"}
Oct 30 21:37:56.097483 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:56.097461    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d487d746-hf6pr" event={"ID":"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e","Type":"ContainerStarted","Data":"ba9877f36e4ecd30976644ec5fd958ced7ab0b76ac57b9584f3b461962fce03c"}
Oct 30 21:37:56.097559 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:56.097503    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d487d746-hf6pr" event={"ID":"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e","Type":"ContainerStarted","Data":"574c8ccb000cf04aae3ef1c987748b7ad799355055cff9bf263ec7f309bb5107"}
Oct 30 21:37:56.097628 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:56.097611    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:37:56.260705 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:56.260673    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:56.263044 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:56.263020    4871 patch_prober.go:29] interesting pod/router-default-57f669cfc9-d755h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld
Oct 30 21:37:56.263044 ip-10-0-139-46 kubenswrapper[4871]: [-]has-synced failed: reason withheld
Oct 30 21:37:56.263044 ip-10-0-139-46 kubenswrapper[4871]: [+]process-running ok
Oct 30 21:37:56.263044 ip-10-0-139-46 kubenswrapper[4871]: healthz check failed
Oct 30 21:37:56.263232 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:56.263068    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-57f669cfc9-d755h" podUID="e0535562-96d7-4fc4-a5d5-ad4cc785080e" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Oct 30 21:37:57.052419 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:57.051631    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-6q6pm"
Oct 30 21:37:57.261687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:57.261652    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:58.107995 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:58.107955    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l9kxl" event={"ID":"7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2","Type":"ContainerStarted","Data":"b1e8a22c6832dbcce700caabaf539cba3a0b6aa2b7c38af609548f7e5474bba2"}
Oct 30 21:37:58.108507 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:58.108053    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l9kxl" event={"ID":"7b0aa12e-a4c2-4f25-96f0-5c2272f8caf2","Type":"ContainerStarted","Data":"bbe313b5d87e73468860c34f5abe9f20807bda1d611632dcdcb0bdb7f5285e86"}
Oct 30 21:37:58.109849 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:58.109825    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-h6hzg" event={"ID":"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee","Type":"ContainerStarted","Data":"033fc6cf572a128826fe2ef689e0d00d3a23be12ff17ec6e269a247002931179"}
Oct 30 21:37:58.111378 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:58.111346    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-5lpq9" event={"ID":"a099b798-61cd-4306-85ba-85c7deb25a18","Type":"ContainerStarted","Data":"8a86ec95eb6aeb51b0da134c412cbbb939c0aa1de019f0cf88c9b2ec4ebba71a"}
Oct 30 21:37:58.111568 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:58.111546    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:58.112940 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:58.112920    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-57f669cfc9-d755h"
Oct 30 21:37:59.118168 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:59.118119    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q2pns" event={"ID":"5cf90653-8f33-4bb2-b622-1dae9144a9fe","Type":"ContainerStarted","Data":"d4de66b89fecdfbc3ea48cd8b4a73ec45affe49fc47759eb4b8c448c5068def3"}
Oct 30 21:37:59.118630 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:59.118201    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q2pns" event={"ID":"5cf90653-8f33-4bb2-b622-1dae9144a9fe","Type":"ContainerStarted","Data":"b0a3e3ae452872a77a056fb95e1d6fbf7aff84e4b4d88931d71a5f3e07fc8668"}
Oct 30 21:37:59.279164 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:59.279130    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:59.429072 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:37:59.428979    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:00.079135 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:00.079101    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"]
Oct 30 21:38:00.124793 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:00.124756    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-h6hzg" event={"ID":"565e807a-a9c3-4f1b-8de0-9ff4d713a1ee","Type":"ContainerStarted","Data":"98c0f6cc4be5c73e20e16532719131003de70b66fc7488228ea73071ba276b98"}
Oct 30 21:38:00.397959 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:00.397916    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:00.614104 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:00.614066    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:01.931554 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:01.931519    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:02.121562 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:02.121527    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:04.157549 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:04.157517    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:04.344478 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:04.344444    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:04.502044 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:04.501962    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:04.677230 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:04.677197    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:05.570396 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:38:05.570354    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:38:08.076201 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:08.076169    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:08.227527 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:08.227492    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:08.376315 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:08.376283    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:08.737407 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:08.737310    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:08.886579 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:08.886543    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:09.507846 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:09.507785    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:09.654560 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:09.654526    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:10.085274 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:10.085221    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:38:10.402831 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:10.402793    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:10.562900 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:10.562858    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:10.755383 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:10.755300    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:10.912369 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:10.912329    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:11.077957 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:11.077878    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:11.202963 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:11.202930    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5wmsz"]
Oct 30 21:38:11.252619 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:11.252588    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5wmsz"
Oct 30 21:38:11.255683 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:11.255660    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:11.353927 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:11.353841    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zf5wc\""
Oct 30 21:38:11.353927 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:11.353862    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 21:38:11.353927 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:11.353839    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 21:38:11.401008 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:11.400978    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 21:38:11.401179 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:11.401035    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 21:38:11.408404 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:11.408373    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70fe39fd-b378-432b-b3bf-4754d342adfb-metrics-client-ca\") pod \"node-exporter-5wmsz\" (UID: \"70fe39fd-b378-432b-b3bf-4754d342adfb\") " pod="openshift-monitoring/node-exporter-5wmsz"
Oct 30 21:38:11.408536 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:11.408417    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/70fe39fd-b378-432b-b3bf-4754d342adfb-node-exporter-wtmp\") pod \"node-exporter-5wmsz\" (UID: \"70fe39fd-b378-432b-b3bf-4754d342adfb\") " pod="openshift-monitoring/node-exporter-5wmsz"
Oct 30 21:38:11.408536 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:11.408444    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/70fe39fd-b378-432b-b3bf-4754d342adfb-node-exporter-accelerators-collector-config\") pod \"node-exporter-5wmsz\" (UID: \"70fe39fd-b378-432b-b3bf-4754d342adfb\") " pod="openshift-monitoring/node-exporter-5wmsz"
Oct 30 21:38:11.408611 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:11.408549    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/70fe39fd-b378-432b-b3bf-4754d342adfb-sys\") pod \"node-exporter-5wmsz\" (UID: \"70fe39fd-b378-432b-b3bf-4754d342adfb\") " pod="openshift-monitoring/node-exporter-5wmsz"
Oct 30 21:38:11.408690 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:11.408668    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/70fe39fd-b378-432b-b3bf-4754d342adfb-root\") pod \"node-exporter-5wmsz\" (UID: \"70fe39fd-b378-432b-b3bf-4754d342adfb\") " pod="openshift-monitoring/node-exporter-5wmsz"
Oct 30 21:38:11.408739 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:11.408704    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6qfq\" (UniqueName: \"kubernetes.io/projected/70fe39fd-b378-432b-b3bf-4754d342adfb-kube-api-access-b6qfq\") pod \"node-exporter-5wmsz\" (UID: \"70fe39fd-b378-432b-b3bf-4754d342adfb\") " pod="openshift-monitoring/node-exporter-5wmsz"
Oct 30 21:38:11.408739 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:11.408727    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/70fe39fd-b378-432b-b3bf-4754d342adfb-node-exporter-textfile\") pod \"node-exporter-5wmsz\" (UID: \"70fe39fd-b378-432b-b3bf-4754d342adfb\") " pod="openshift-monitoring/node-exporter-5wmsz"
Oct 30 21:38:11.408804 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:11.408771    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/70fe39fd-b378-432b-b3bf-4754d342adfb-node-exporter-tls\") pod \"node-exporter-5wmsz\" (UID: \"70fe39fd-b378-432b-b3bf-4754d342adfb\") " pod="openshift-monitoring/node-exporter-5wmsz"
Oct 30 21:38:11.408837 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:11.408807    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/70fe39fd-b378-432b-b3bf-4754d342adfb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5wmsz\" (UID: \"70fe39fd-b378-432b-b3bf-4754d342adfb\") " pod="openshift-monitoring/node-exporter-5wmsz"
Oct 30 21:38:11.441088 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:11.441055    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:11.509837 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:11.509800    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/70fe39fd-b378-432b-b3bf-4754d342adfb-root\") pod \"node-exporter-5wmsz\" (UID: \"70fe39fd-b378-432b-b3bf-4754d342adfb\") " pod="openshift-monitoring/node-exporter-5wmsz"
Oct 30 21:38:11.509837 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:11.509839    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6qfq\" (UniqueName: \"kubernetes.io/projected/70fe39fd-b378-432b-b3bf-4754d342adfb-kube-api-access-b6qfq\") pod \"node-exporter-5wmsz\" (UID: \"70fe39fd-b378-432b-b3bf-4754d342adfb\") " pod="openshift-monitoring/node-exporter-5wmsz"
Oct 30 21:38:11.510095 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:11.509860    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/70fe39fd-b378-432b-b3bf-4754d342adfb-node-exporter-textfile\") pod \"node-exporter-5wmsz\" (UID: \"70fe39fd-b378-432b-b3bf-4754d342adfb\") " pod="openshift-monitoring/node-exporter-5wmsz"
Oct 30 21:38:11.510095 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:11.509878    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/70fe39fd-b378-432b-b3bf-4754d342adfb-node-exporter-tls\") pod \"node-exporter-5wmsz\" (UID: \"70fe39fd-b378-432b-b3bf-4754d342adfb\") " pod="openshift-monitoring/node-exporter-5wmsz"
Oct 30 21:38:11.510095 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:11.509897    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/70fe39fd-b378-432b-b3bf-4754d342adfb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5wmsz\" (UID: \"70fe39fd-b378-432b-b3bf-4754d342adfb\") " pod="openshift-monitoring/node-exporter-5wmsz"
Oct 30 21:38:11.510095 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:11.509910    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/70fe39fd-b378-432b-b3bf-4754d342adfb-root\") pod \"node-exporter-5wmsz\" (UID: \"70fe39fd-b378-432b-b3bf-4754d342adfb\") " pod="openshift-monitoring/node-exporter-5wmsz"
Oct 30 21:38:11.510095 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:11.509946    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70fe39fd-b378-432b-b3bf-4754d342adfb-metrics-client-ca\") pod \"node-exporter-5wmsz\" (UID: \"70fe39fd-b378-432b-b3bf-4754d342adfb\") " pod="openshift-monitoring/node-exporter-5wmsz"
Oct 30 21:38:11.510095 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:11.509980    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/70fe39fd-b378-432b-b3bf-4754d342adfb-node-exporter-wtmp\") pod \"node-exporter-5wmsz\" (UID: \"70fe39fd-b378-432b-b3bf-4754d342adfb\") " pod="openshift-monitoring/node-exporter-5wmsz"
Oct 30 21:38:11.510095 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:11.510007    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/70fe39fd-b378-432b-b3bf-4754d342adfb-node-exporter-accelerators-collector-config\") pod \"node-exporter-5wmsz\" (UID: \"70fe39fd-b378-432b-b3bf-4754d342adfb\") " pod="openshift-monitoring/node-exporter-5wmsz"
Oct 30 21:38:11.510095 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:11.510028    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/70fe39fd-b378-432b-b3bf-4754d342adfb-sys\") pod \"node-exporter-5wmsz\" (UID: \"70fe39fd-b378-432b-b3bf-4754d342adfb\") " pod="openshift-monitoring/node-exporter-5wmsz"
Oct 30 21:38:11.510426 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:11.510102    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/70fe39fd-b378-432b-b3bf-4754d342adfb-sys\") pod \"node-exporter-5wmsz\" (UID: \"70fe39fd-b378-432b-b3bf-4754d342adfb\") " pod="openshift-monitoring/node-exporter-5wmsz"
Oct 30 21:38:11.510426 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:11.510143    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/70fe39fd-b378-432b-b3bf-4754d342adfb-node-exporter-wtmp\") pod \"node-exporter-5wmsz\" (UID: \"70fe39fd-b378-432b-b3bf-4754d342adfb\") " pod="openshift-monitoring/node-exporter-5wmsz"
Oct 30 21:38:11.510426 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:11.510287    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/70fe39fd-b378-432b-b3bf-4754d342adfb-node-exporter-textfile\") pod \"node-exporter-5wmsz\" (UID: \"70fe39fd-b378-432b-b3bf-4754d342adfb\") " pod="openshift-monitoring/node-exporter-5wmsz"
Oct 30 21:38:11.510638 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:11.510615    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/70fe39fd-b378-432b-b3bf-4754d342adfb-node-exporter-accelerators-collector-config\") pod \"node-exporter-5wmsz\" (UID: \"70fe39fd-b378-432b-b3bf-4754d342adfb\") " pod="openshift-monitoring/node-exporter-5wmsz"
Oct 30 21:38:11.510704 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:11.510617    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70fe39fd-b378-432b-b3bf-4754d342adfb-metrics-client-ca\") pod \"node-exporter-5wmsz\" (UID: \"70fe39fd-b378-432b-b3bf-4754d342adfb\") " pod="openshift-monitoring/node-exporter-5wmsz"
Oct 30 21:38:11.512397 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:11.512373    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/70fe39fd-b378-432b-b3bf-4754d342adfb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5wmsz\" (UID: \"70fe39fd-b378-432b-b3bf-4754d342adfb\") " pod="openshift-monitoring/node-exporter-5wmsz"
Oct 30 21:38:11.512492 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:11.512446    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/70fe39fd-b378-432b-b3bf-4754d342adfb-node-exporter-tls\") pod \"node-exporter-5wmsz\" (UID: \"70fe39fd-b378-432b-b3bf-4754d342adfb\") " pod="openshift-monitoring/node-exporter-5wmsz"
Oct 30 21:38:11.572429 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:11.572390    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6qfq\" (UniqueName: \"kubernetes.io/projected/70fe39fd-b378-432b-b3bf-4754d342adfb-kube-api-access-b6qfq\") pod \"node-exporter-5wmsz\" (UID: \"70fe39fd-b378-432b-b3bf-4754d342adfb\") " pod="openshift-monitoring/node-exporter-5wmsz"
Oct 30 21:38:11.604885 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:11.604807    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5wmsz"
Oct 30 21:38:11.640074 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:11.640045    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:11.645125 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:38:11.645098    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70fe39fd_b378_432b_b3bf_4754d342adfb.slice/crio-15506f4f0a56765f5cb1dca4af4a014333e1eab9814eca6cb18a90226476c0be WatchSource:0}: Error finding container 15506f4f0a56765f5cb1dca4af4a014333e1eab9814eca6cb18a90226476c0be: Status 404 returned error can't find the container with id 15506f4f0a56765f5cb1dca4af4a014333e1eab9814eca6cb18a90226476c0be
Oct 30 21:38:11.794648 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:11.794612    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:12.169259 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:12.169210    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5wmsz" event={"ID":"70fe39fd-b378-432b-b3bf-4754d342adfb","Type":"ContainerStarted","Data":"15506f4f0a56765f5cb1dca4af4a014333e1eab9814eca6cb18a90226476c0be"}
Oct 30 21:38:12.281821 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:12.281788    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:12.436990 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:12.436913    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:12.678287 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:12.678255    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:12.878025 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:12.877978    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:13.328071 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:13.327984    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-wpc6f"
Oct 30 21:38:13.455713 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:13.455674    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:14.179990 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:14.179955    4871 generic.go:356] "Generic (PLEG): container finished" podID="70fe39fd-b378-432b-b3bf-4754d342adfb" containerID="f1e9b01b43cc58cb0486153f76a3f263dee25b7deb91bd1c681f2c851f5a0fcd" exitCode=0
Oct 30 21:38:14.180469 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:14.180005    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5wmsz" event={"ID":"70fe39fd-b378-432b-b3bf-4754d342adfb","Type":"ContainerDied","Data":"f1e9b01b43cc58cb0486153f76a3f263dee25b7deb91bd1c681f2c851f5a0fcd"}
Oct 30 21:38:15.186734 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:15.186695    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5wmsz" event={"ID":"70fe39fd-b378-432b-b3bf-4754d342adfb","Type":"ContainerStarted","Data":"cfd98e79a6e9258cd090ed416c7a1aef91197aae2e8d63d27218cf2c7c942962"}
Oct 30 21:38:15.187205 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:15.186762    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5wmsz" event={"ID":"70fe39fd-b378-432b-b3bf-4754d342adfb","Type":"ContainerStarted","Data":"f82b4a2a45677624400b0ee4b9beca95a29535460cd88afd6572036f02a8445b"}
Oct 30 21:38:15.307073 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:15.307018    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5wmsz" podStartSLOduration=2.084436202 podStartE2EDuration="4.307001065s" totalImagesPullingTime="2.132017508s" totalInitContainerRuntime="90.547355ms" isStatefulPod=true podCreationTimestamp="2026-10-30 21:38:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:38:15.305712504 +0000 UTC m=+37.483201122" watchObservedRunningTime="2026-10-30 21:38:15.307001065 +0000 UTC m=+37.484489673"
Oct 30 21:38:15.445808 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:15.445721    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:15.635522 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:15.635491    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:15.822632 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:15.822557    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:15.994571 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:15.994537    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:16.275665 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:16.275631    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:16.537745 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:16.537671    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:16.712555 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:16.712524    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:17.106553 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:17.106525    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:38:18.290157 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:18.290084    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:18.457946 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:18.457910    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:20.358922 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:20.358881    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7d487d746-hf6pr"]
Oct 30 21:38:20.431798 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:20.431765    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:20.731399 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:20.731342    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:21.054721 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:21.054624    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:21.337973 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:21.337870    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:21.675167 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:21.675132    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:21.748304 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:21.748225    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-9bb676567-hz5w8"]
Oct 30 21:38:21.750636 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:21.750613    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9bb676567-hz5w8"
Oct 30 21:38:21.800605 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:21.800572    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f619293a-1e5d-4027-a30b-4eec092b0b1e-service-ca\") pod \"console-9bb676567-hz5w8\" (UID: \"f619293a-1e5d-4027-a30b-4eec092b0b1e\") " pod="openshift-console/console-9bb676567-hz5w8"
Oct 30 21:38:21.800605 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:21.800610    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-582ld\" (UniqueName: \"kubernetes.io/projected/f619293a-1e5d-4027-a30b-4eec092b0b1e-kube-api-access-582ld\") pod \"console-9bb676567-hz5w8\" (UID: \"f619293a-1e5d-4027-a30b-4eec092b0b1e\") " pod="openshift-console/console-9bb676567-hz5w8"
Oct 30 21:38:21.800877 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:21.800634    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f619293a-1e5d-4027-a30b-4eec092b0b1e-console-serving-cert\") pod \"console-9bb676567-hz5w8\" (UID: \"f619293a-1e5d-4027-a30b-4eec092b0b1e\") " pod="openshift-console/console-9bb676567-hz5w8"
Oct 30 21:38:21.800877 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:21.800693    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f619293a-1e5d-4027-a30b-4eec092b0b1e-tmp\") pod \"console-9bb676567-hz5w8\" (UID: \"f619293a-1e5d-4027-a30b-4eec092b0b1e\") " pod="openshift-console/console-9bb676567-hz5w8"
Oct 30 21:38:21.800877 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:21.800756    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f619293a-1e5d-4027-a30b-4eec092b0b1e-trusted-ca-bundle\") pod \"console-9bb676567-hz5w8\" (UID: \"f619293a-1e5d-4027-a30b-4eec092b0b1e\") " pod="openshift-console/console-9bb676567-hz5w8"
Oct 30 21:38:21.800877 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:21.800788    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f619293a-1e5d-4027-a30b-4eec092b0b1e-oauth-serving-cert\") pod \"console-9bb676567-hz5w8\" (UID: \"f619293a-1e5d-4027-a30b-4eec092b0b1e\") " pod="openshift-console/console-9bb676567-hz5w8"
Oct 30 21:38:21.801045 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:21.800903    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f619293a-1e5d-4027-a30b-4eec092b0b1e-console-config\") pod \"console-9bb676567-hz5w8\" (UID: \"f619293a-1e5d-4027-a30b-4eec092b0b1e\") " pod="openshift-console/console-9bb676567-hz5w8"
Oct 30 21:38:21.801045 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:21.800953    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f619293a-1e5d-4027-a30b-4eec092b0b1e-console-oauth-config\") pod \"console-9bb676567-hz5w8\" (UID: \"f619293a-1e5d-4027-a30b-4eec092b0b1e\") " pod="openshift-console/console-9bb676567-hz5w8"
Oct 30 21:38:21.851512 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:21.851456    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 30 21:38:21.851512 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:21.851474    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-czbd6\""
Oct 30 21:38:21.851739 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:21.851542    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 30 21:38:21.851739 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:21.851552    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 30 21:38:21.901427 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:21.901369    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f619293a-1e5d-4027-a30b-4eec092b0b1e-console-config\") pod \"console-9bb676567-hz5w8\" (UID: \"f619293a-1e5d-4027-a30b-4eec092b0b1e\") " pod="openshift-console/console-9bb676567-hz5w8"
Oct 30 21:38:21.901427 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:21.901434    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f619293a-1e5d-4027-a30b-4eec092b0b1e-console-oauth-config\") pod \"console-9bb676567-hz5w8\" (UID: \"f619293a-1e5d-4027-a30b-4eec092b0b1e\") " pod="openshift-console/console-9bb676567-hz5w8"
Oct 30 21:38:21.901650 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:21.901470    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f619293a-1e5d-4027-a30b-4eec092b0b1e-service-ca\") pod \"console-9bb676567-hz5w8\" (UID: \"f619293a-1e5d-4027-a30b-4eec092b0b1e\") " pod="openshift-console/console-9bb676567-hz5w8"
Oct 30 21:38:21.901650 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:21.901490    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-582ld\" (UniqueName: \"kubernetes.io/projected/f619293a-1e5d-4027-a30b-4eec092b0b1e-kube-api-access-582ld\") pod \"console-9bb676567-hz5w8\" (UID: \"f619293a-1e5d-4027-a30b-4eec092b0b1e\") " pod="openshift-console/console-9bb676567-hz5w8"
Oct 30 21:38:21.901650 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:21.901510    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f619293a-1e5d-4027-a30b-4eec092b0b1e-console-serving-cert\") pod \"console-9bb676567-hz5w8\" (UID: \"f619293a-1e5d-4027-a30b-4eec092b0b1e\") " pod="openshift-console/console-9bb676567-hz5w8"
Oct 30 21:38:21.901650 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:21.901527    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f619293a-1e5d-4027-a30b-4eec092b0b1e-tmp\") pod \"console-9bb676567-hz5w8\" (UID: \"f619293a-1e5d-4027-a30b-4eec092b0b1e\") " pod="openshift-console/console-9bb676567-hz5w8"
Oct 30 21:38:21.901650 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:21.901590    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f619293a-1e5d-4027-a30b-4eec092b0b1e-trusted-ca-bundle\") pod \"console-9bb676567-hz5w8\" (UID: \"f619293a-1e5d-4027-a30b-4eec092b0b1e\") " pod="openshift-console/console-9bb676567-hz5w8"
Oct 30 21:38:21.901650 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:21.901619    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f619293a-1e5d-4027-a30b-4eec092b0b1e-oauth-serving-cert\") pod \"console-9bb676567-hz5w8\" (UID: \"f619293a-1e5d-4027-a30b-4eec092b0b1e\") " pod="openshift-console/console-9bb676567-hz5w8"
Oct 30 21:38:21.901946 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:21.901916    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f619293a-1e5d-4027-a30b-4eec092b0b1e-tmp\") pod \"console-9bb676567-hz5w8\" (UID: \"f619293a-1e5d-4027-a30b-4eec092b0b1e\") " pod="openshift-console/console-9bb676567-hz5w8"
Oct 30 21:38:21.903979 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:21.903951    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f619293a-1e5d-4027-a30b-4eec092b0b1e-console-oauth-config\") pod \"console-9bb676567-hz5w8\" (UID: \"f619293a-1e5d-4027-a30b-4eec092b0b1e\") " pod="openshift-console/console-9bb676567-hz5w8"
Oct 30 21:38:21.904106 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:21.904036    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f619293a-1e5d-4027-a30b-4eec092b0b1e-console-serving-cert\") pod \"console-9bb676567-hz5w8\" (UID: \"f619293a-1e5d-4027-a30b-4eec092b0b1e\") " pod="openshift-console/console-9bb676567-hz5w8"
Oct 30 21:38:21.952994 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:21.952904    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 30 21:38:21.953178 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:21.953112    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 30 21:38:21.953178 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:21.953145    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 30 21:38:21.953178 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:21.953190    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 30 21:38:21.953367 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:21.953210    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 30 21:38:21.957739 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:21.957719    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9bb676567-hz5w8"]
Oct 30 21:38:21.962551 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:21.962526    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f619293a-1e5d-4027-a30b-4eec092b0b1e-service-ca\") pod \"console-9bb676567-hz5w8\" (UID: \"f619293a-1e5d-4027-a30b-4eec092b0b1e\") " pod="openshift-console/console-9bb676567-hz5w8"
Oct 30 21:38:21.962649 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:21.962602    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f619293a-1e5d-4027-a30b-4eec092b0b1e-oauth-serving-cert\") pod \"console-9bb676567-hz5w8\" (UID: \"f619293a-1e5d-4027-a30b-4eec092b0b1e\") " pod="openshift-console/console-9bb676567-hz5w8"
Oct 30 21:38:21.962725 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:21.962702    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f619293a-1e5d-4027-a30b-4eec092b0b1e-console-config\") pod \"console-9bb676567-hz5w8\" (UID: \"f619293a-1e5d-4027-a30b-4eec092b0b1e\") " pod="openshift-console/console-9bb676567-hz5w8"
Oct 30 21:38:21.962953 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:21.962936    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f619293a-1e5d-4027-a30b-4eec092b0b1e-trusted-ca-bundle\") pod \"console-9bb676567-hz5w8\" (UID: \"f619293a-1e5d-4027-a30b-4eec092b0b1e\") " pod="openshift-console/console-9bb676567-hz5w8"
Oct 30 21:38:21.972965 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:21.972933    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-582ld\" (UniqueName: \"kubernetes.io/projected/f619293a-1e5d-4027-a30b-4eec092b0b1e-kube-api-access-582ld\") pod \"console-9bb676567-hz5w8\" (UID: \"f619293a-1e5d-4027-a30b-4eec092b0b1e\") " pod="openshift-console/console-9bb676567-hz5w8"
Oct 30 21:38:22.061646 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:22.061605    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9bb676567-hz5w8"
Oct 30 21:38:22.455524 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:22.455489    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9bb676567-hz5w8"]
Oct 30 21:38:22.457566 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:38:22.457530    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf619293a_1e5d_4027_a30b_4eec092b0b1e.slice/crio-e2f6d63ceaa23fd668f13ae333e7dc0739a76ff36a1c469c551839d44954b4a4 WatchSource:0}: Error finding container e2f6d63ceaa23fd668f13ae333e7dc0739a76ff36a1c469c551839d44954b4a4: Status 404 returned error can't find the container with id e2f6d63ceaa23fd668f13ae333e7dc0739a76ff36a1c469c551839d44954b4a4
Oct 30 21:38:22.596685 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:22.596646    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:22.852409 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:22.852325    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:23.216942 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:23.216904    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9bb676567-hz5w8" event={"ID":"f619293a-1e5d-4027-a30b-4eec092b0b1e","Type":"ContainerStarted","Data":"e2f6d63ceaa23fd668f13ae333e7dc0739a76ff36a1c469c551839d44954b4a4"}
Oct 30 21:38:25.087398 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:25.087362    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:25.098984 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:25.098929    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq" podUID="0dca73b7-6526-48ac-a76e-70e441debc00" containerName="registry" containerID="cri-o://a5c9c0b4ad24703407193b74a120de5f0cb8bdedfc45aae069e612b364c1171e" gracePeriod=30
Oct 30 21:38:25.228488 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:25.228442    4871 generic.go:356] "Generic (PLEG): container finished" podID="0dca73b7-6526-48ac-a76e-70e441debc00" containerID="a5c9c0b4ad24703407193b74a120de5f0cb8bdedfc45aae069e612b364c1171e" exitCode=0
Oct 30 21:38:25.228671 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:25.228521    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq" event={"ID":"0dca73b7-6526-48ac-a76e-70e441debc00","Type":"ContainerDied","Data":"a5c9c0b4ad24703407193b74a120de5f0cb8bdedfc45aae069e612b364c1171e"}
Oct 30 21:38:25.372008 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:25.371968    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:38:25.428910 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:25.428874    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-kube-api-access-g8hlw\" (UniqueName: \"kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-kube-api-access-g8hlw\") pod \"0dca73b7-6526-48ac-a76e-70e441debc00\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") "
Oct 30 21:38:25.429091 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:25.428940    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0dca73b7-6526-48ac-a76e-70e441debc00-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0dca73b7-6526-48ac-a76e-70e441debc00-trusted-ca\") pod \"0dca73b7-6526-48ac-a76e-70e441debc00\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") "
Oct 30 21:38:25.429091 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:25.428970    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0dca73b7-6526-48ac-a76e-70e441debc00-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0dca73b7-6526-48ac-a76e-70e441debc00-registry-certificates\") pod \"0dca73b7-6526-48ac-a76e-70e441debc00\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") "
Oct 30 21:38:25.429336 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:25.429308    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-registry-tls\" (UniqueName: \"kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-registry-tls\") pod \"0dca73b7-6526-48ac-a76e-70e441debc00\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") "
Oct 30 21:38:25.429587 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:25.429360    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-bound-sa-token\") pod \"0dca73b7-6526-48ac-a76e-70e441debc00\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") "
Oct 30 21:38:25.429587 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:25.429417    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0dca73b7-6526-48ac-a76e-70e441debc00-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0dca73b7-6526-48ac-a76e-70e441debc00-installation-pull-secrets\") pod \"0dca73b7-6526-48ac-a76e-70e441debc00\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") "
Oct 30 21:38:25.429587 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:25.429462    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0dca73b7-6526-48ac-a76e-70e441debc00-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0dca73b7-6526-48ac-a76e-70e441debc00-ca-trust-extracted\") pod \"0dca73b7-6526-48ac-a76e-70e441debc00\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") "
Oct 30 21:38:25.429587 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:25.429512    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0dca73b7-6526-48ac-a76e-70e441debc00-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0dca73b7-6526-48ac-a76e-70e441debc00-image-registry-private-configuration\") pod \"0dca73b7-6526-48ac-a76e-70e441debc00\" (UID: \"0dca73b7-6526-48ac-a76e-70e441debc00\") "
Oct 30 21:38:25.429587 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:25.429551    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0dca73b7-6526-48ac-a76e-70e441debc00-registry-certificates" pod "0dca73b7-6526-48ac-a76e-70e441debc00" (UID: "0dca73b7-6526-48ac-a76e-70e441debc00"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:38:25.429865 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:25.429655    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0dca73b7-6526-48ac-a76e-70e441debc00-trusted-ca" pod "0dca73b7-6526-48ac-a76e-70e441debc00" (UID: "0dca73b7-6526-48ac-a76e-70e441debc00"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:38:25.430109 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:25.430086    4871 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0dca73b7-6526-48ac-a76e-70e441debc00-registry-certificates\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:38:25.432137 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:25.432108    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0dca73b7-6526-48ac-a76e-70e441debc00-image-registry-private-configuration" pod "0dca73b7-6526-48ac-a76e-70e441debc00" (UID: "0dca73b7-6526-48ac-a76e-70e441debc00"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:38:25.432304 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:25.432225    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0dca73b7-6526-48ac-a76e-70e441debc00-installation-pull-secrets" pod "0dca73b7-6526-48ac-a76e-70e441debc00" (UID: "0dca73b7-6526-48ac-a76e-70e441debc00"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:38:25.432304 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:25.432229    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-registry-tls" pod "0dca73b7-6526-48ac-a76e-70e441debc00" (UID: "0dca73b7-6526-48ac-a76e-70e441debc00"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:25.432428 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:25.432298    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-bound-sa-token" pod "0dca73b7-6526-48ac-a76e-70e441debc00" (UID: "0dca73b7-6526-48ac-a76e-70e441debc00"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:25.432483 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:25.432421    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-kube-api-access-g8hlw" pod "0dca73b7-6526-48ac-a76e-70e441debc00" (UID: "0dca73b7-6526-48ac-a76e-70e441debc00"). InnerVolumeSpecName "kube-api-access-g8hlw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:25.437545 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:25.437495    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0dca73b7-6526-48ac-a76e-70e441debc00-ca-trust-extracted" pod "0dca73b7-6526-48ac-a76e-70e441debc00" (UID: "0dca73b7-6526-48ac-a76e-70e441debc00"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:25.530733 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:25.530690    4871 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0dca73b7-6526-48ac-a76e-70e441debc00-installation-pull-secrets\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:38:25.530733 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:25.530728    4871 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0dca73b7-6526-48ac-a76e-70e441debc00-ca-trust-extracted\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:38:25.530733 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:25.530740    4871 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0dca73b7-6526-48ac-a76e-70e441debc00-image-registry-private-configuration\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:38:25.531027 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:25.530749    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8hlw\" (UniqueName: \"kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-kube-api-access-g8hlw\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:38:25.531027 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:25.530762    4871 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0dca73b7-6526-48ac-a76e-70e441debc00-trusted-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:38:25.531027 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:25.530776    4871 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-registry-tls\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:38:25.531027 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:25.530789    4871 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0dca73b7-6526-48ac-a76e-70e441debc00-bound-sa-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:38:26.233875 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:26.233779    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq" event={"ID":"0dca73b7-6526-48ac-a76e-70e441debc00","Type":"ContainerDied","Data":"cd1aab9d007f228e04db2b8ee9e607316cb2624c20cab7fc88dc3a1d2d1a7d25"}
Oct 30 21:38:26.233875 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:26.233874    4871 scope.go:122] "RemoveContainer" containerID="a5c9c0b4ad24703407193b74a120de5f0cb8bdedfc45aae069e612b364c1171e"
Oct 30 21:38:26.234457 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:26.233802    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"
Oct 30 21:38:26.235546 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:26.235513    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9bb676567-hz5w8" event={"ID":"f619293a-1e5d-4027-a30b-4eec092b0b1e","Type":"ContainerStarted","Data":"7fbff780f4a214416f2d3a3127f003e9c45a01d1ac1e547d51ffb352bacb9ff7"}
Oct 30 21:38:26.399737 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:26.399670    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-9bb676567-hz5w8" podStartSLOduration=2.244207093 podStartE2EDuration="5.399651931s" totalImagesPullingTime="3.155444838s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:38:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:38:26.397903268 +0000 UTC m=+48.575391898" watchObservedRunningTime="2026-10-30 21:38:26.399651931 +0000 UTC m=+48.577140561"
Oct 30 21:38:26.505839 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:26.505757    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"]
Oct 30 21:38:26.512972 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:26.512943    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-55fbc4d8d7-w4mrq"]
Oct 30 21:38:27.176430 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:27.176395    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9bb676567-hz5w8"]
Oct 30 21:38:27.958999 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:27.958964    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0dca73b7-6526-48ac-a76e-70e441debc00" path="/var/lib/kubelet/pods/0dca73b7-6526-48ac-a76e-70e441debc00/volumes"
Oct 30 21:38:28.019511 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:28.019474    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:28.415100 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:28.415067    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:28.652525 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:28.652490    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:28.870450 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:28.870417    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:30.965247 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:30.965214    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:31.338369 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:31.338288    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:31.990449 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:31.990410    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:32.062365 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:32.062322    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-9bb676567-hz5w8"
Oct 30 21:38:32.231696 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:32.231662    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:35.567391 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:38:35.567351    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:38:44.585043 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:44.585004    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:44.862103 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:44.862060    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:45.378627 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:45.378590    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7d487d746-hf6pr" podUID="30931e26-479e-4ddb-8ce3-9b2b52dc0e8e" containerName="registry" containerID="cri-o://ba9877f36e4ecd30976644ec5fd958ced7ab0b76ac57b9584f3b461962fce03c" gracePeriod=30
Oct 30 21:38:45.645024 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:45.644999    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:38:45.703911 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:45.703878    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-certificates\") pod \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") "
Oct 30 21:38:45.704084 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:45.703954    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-kube-api-access-d49lh\" (UniqueName: \"kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-kube-api-access-d49lh\") pod \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") "
Oct 30 21:38:45.704084 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:45.703983    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-tls\" (UniqueName: \"kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-tls\") pod \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") "
Oct 30 21:38:45.704084 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:45.704014    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-ca-trust-extracted\") pod \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") "
Oct 30 21:38:45.704084 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:45.704043    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-trusted-ca\") pod \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") "
Oct 30 21:38:45.704084 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:45.704072    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-bound-sa-token\") pod \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") "
Oct 30 21:38:45.704372 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:45.704119    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-installation-pull-secrets\") pod \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") "
Oct 30 21:38:45.704372 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:45.704150    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-image-registry-private-configuration\") pod \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\" (UID: \"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e\") "
Oct 30 21:38:45.704528 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:45.704500    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-certificates" pod "30931e26-479e-4ddb-8ce3-9b2b52dc0e8e" (UID: "30931e26-479e-4ddb-8ce3-9b2b52dc0e8e"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:38:45.705372 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:45.705325    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-trusted-ca" pod "30931e26-479e-4ddb-8ce3-9b2b52dc0e8e" (UID: "30931e26-479e-4ddb-8ce3-9b2b52dc0e8e"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:38:45.706431 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:45.706407    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-tls" pod "30931e26-479e-4ddb-8ce3-9b2b52dc0e8e" (UID: "30931e26-479e-4ddb-8ce3-9b2b52dc0e8e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:45.706869 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:45.706844    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-bound-sa-token" pod "30931e26-479e-4ddb-8ce3-9b2b52dc0e8e" (UID: "30931e26-479e-4ddb-8ce3-9b2b52dc0e8e"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:45.707040 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:45.707013    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-image-registry-private-configuration" pod "30931e26-479e-4ddb-8ce3-9b2b52dc0e8e" (UID: "30931e26-479e-4ddb-8ce3-9b2b52dc0e8e"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:38:45.707247 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:45.707211    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-installation-pull-secrets" pod "30931e26-479e-4ddb-8ce3-9b2b52dc0e8e" (UID: "30931e26-479e-4ddb-8ce3-9b2b52dc0e8e"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:38:45.708019 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:45.707994    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-kube-api-access-d49lh" pod "30931e26-479e-4ddb-8ce3-9b2b52dc0e8e" (UID: "30931e26-479e-4ddb-8ce3-9b2b52dc0e8e"). InnerVolumeSpecName "kube-api-access-d49lh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:45.711701 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:45.711672    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-ca-trust-extracted" pod "30931e26-479e-4ddb-8ce3-9b2b52dc0e8e" (UID: "30931e26-479e-4ddb-8ce3-9b2b52dc0e8e"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:45.805612 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:45.805573    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d49lh\" (UniqueName: \"kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-kube-api-access-d49lh\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:38:45.805612 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:45.805605    4871 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-tls\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:38:45.805612 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:45.805614    4871 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-ca-trust-extracted\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:38:45.805841 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:45.805625    4871 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-trusted-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:38:45.805841 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:45.805634    4871 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-bound-sa-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:38:45.805841 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:45.805643    4871 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-installation-pull-secrets\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:38:45.805841 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:45.805652    4871 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-image-registry-private-configuration\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:38:45.805841 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:45.805661    4871 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e-registry-certificates\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:38:46.313690 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:46.313598    4871 generic.go:356] "Generic (PLEG): container finished" podID="30931e26-479e-4ddb-8ce3-9b2b52dc0e8e" containerID="ba9877f36e4ecd30976644ec5fd958ced7ab0b76ac57b9584f3b461962fce03c" exitCode=0
Oct 30 21:38:46.313690 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:46.313649    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d487d746-hf6pr"
Oct 30 21:38:46.313690 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:46.313671    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d487d746-hf6pr" event={"ID":"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e","Type":"ContainerDied","Data":"ba9877f36e4ecd30976644ec5fd958ced7ab0b76ac57b9584f3b461962fce03c"}
Oct 30 21:38:46.313932 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:46.313722    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d487d746-hf6pr" event={"ID":"30931e26-479e-4ddb-8ce3-9b2b52dc0e8e","Type":"ContainerDied","Data":"574c8ccb000cf04aae3ef1c987748b7ad799355055cff9bf263ec7f309bb5107"}
Oct 30 21:38:46.313932 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:46.313749    4871 scope.go:122] "RemoveContainer" containerID="ba9877f36e4ecd30976644ec5fd958ced7ab0b76ac57b9584f3b461962fce03c"
Oct 30 21:38:46.321900 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:46.321875    4871 scope.go:122] "RemoveContainer" containerID="ba9877f36e4ecd30976644ec5fd958ced7ab0b76ac57b9584f3b461962fce03c"
Oct 30 21:38:46.322208 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:38:46.322183    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba9877f36e4ecd30976644ec5fd958ced7ab0b76ac57b9584f3b461962fce03c\": container with ID starting with ba9877f36e4ecd30976644ec5fd958ced7ab0b76ac57b9584f3b461962fce03c not found: ID does not exist" containerID="ba9877f36e4ecd30976644ec5fd958ced7ab0b76ac57b9584f3b461962fce03c"
Oct 30 21:38:46.322304 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:46.322217    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba9877f36e4ecd30976644ec5fd958ced7ab0b76ac57b9584f3b461962fce03c"} err="failed to get container status \"ba9877f36e4ecd30976644ec5fd958ced7ab0b76ac57b9584f3b461962fce03c\": rpc error: code = NotFound desc = could not find container \"ba9877f36e4ecd30976644ec5fd958ced7ab0b76ac57b9584f3b461962fce03c\": container with ID starting with ba9877f36e4ecd30976644ec5fd958ced7ab0b76ac57b9584f3b461962fce03c not found: ID does not exist"
Oct 30 21:38:46.476807 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:46.476774    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7d487d746-hf6pr"]
Oct 30 21:38:46.523890 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:46.523858    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7d487d746-hf6pr"]
Oct 30 21:38:46.958654 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:46.958627    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:47.231489 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:47.231404    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:47.656097 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:47.656064    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:47.958061 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:47.957984    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="30931e26-479e-4ddb-8ce3-9b2b52dc0e8e" path="/var/lib/kubelet/pods/30931e26-479e-4ddb-8ce3-9b2b52dc0e8e/volumes"
Oct 30 21:38:47.964086 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:47.964056    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:49.327418 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:49.327388    4871 generic.go:356] "Generic (PLEG): container finished" podID="37186e06-8d56-4c61-ba10-e0408d2e2fa4" containerID="450b76fbd57f402c2f415a74e460d3495cbf9fa7a9222b7ab243b8e03e41e067" exitCode=0
Oct 30 21:38:49.327800 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:49.327464    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-jgspb" event={"ID":"37186e06-8d56-4c61-ba10-e0408d2e2fa4","Type":"ContainerDied","Data":"450b76fbd57f402c2f415a74e460d3495cbf9fa7a9222b7ab243b8e03e41e067"}
Oct 30 21:38:49.327800 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:49.327788    4871 scope.go:122] "RemoveContainer" containerID="450b76fbd57f402c2f415a74e460d3495cbf9fa7a9222b7ab243b8e03e41e067"
Oct 30 21:38:50.257213 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:50.257177    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:50.332727 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:50.332689    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-jgspb" event={"ID":"37186e06-8d56-4c61-ba10-e0408d2e2fa4","Type":"ContainerStarted","Data":"c4a3072c4f1dba2d3b9ebbcbd02622ccef96a4fce1437c5563d37f985cc37be2"}
Oct 30 21:38:50.474441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:50.474392    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:52.110150 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:52.110117    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:52.337777 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:52.337745    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:53.263915 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:53.263870    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-9bb676567-hz5w8" podUID="f619293a-1e5d-4027-a30b-4eec092b0b1e" containerName="console" containerID="cri-o://7fbff780f4a214416f2d3a3127f003e9c45a01d1ac1e547d51ffb352bacb9ff7" gracePeriod=15
Oct 30 21:38:53.387179 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:53.387132    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:53.522583 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:53.522522    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9bb676567-hz5w8"
Oct 30 21:38:53.624986 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:53.624953    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:53.671553 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:53.671524    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f619293a-1e5d-4027-a30b-4eec092b0b1e-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f619293a-1e5d-4027-a30b-4eec092b0b1e-console-serving-cert\") pod \"f619293a-1e5d-4027-a30b-4eec092b0b1e\" (UID: \"f619293a-1e5d-4027-a30b-4eec092b0b1e\") "
Oct 30 21:38:53.671727 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:53.671563    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f619293a-1e5d-4027-a30b-4eec092b0b1e-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f619293a-1e5d-4027-a30b-4eec092b0b1e-trusted-ca-bundle\") pod \"f619293a-1e5d-4027-a30b-4eec092b0b1e\" (UID: \"f619293a-1e5d-4027-a30b-4eec092b0b1e\") "
Oct 30 21:38:53.671727 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:53.671592    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f619293a-1e5d-4027-a30b-4eec092b0b1e-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f619293a-1e5d-4027-a30b-4eec092b0b1e-oauth-serving-cert\") pod \"f619293a-1e5d-4027-a30b-4eec092b0b1e\" (UID: \"f619293a-1e5d-4027-a30b-4eec092b0b1e\") "
Oct 30 21:38:53.671727 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:53.671639    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f619293a-1e5d-4027-a30b-4eec092b0b1e-service-ca\" (UniqueName: \"kubernetes.io/configmap/f619293a-1e5d-4027-a30b-4eec092b0b1e-service-ca\") pod \"f619293a-1e5d-4027-a30b-4eec092b0b1e\" (UID: \"f619293a-1e5d-4027-a30b-4eec092b0b1e\") "
Oct 30 21:38:53.671727 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:53.671661    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f619293a-1e5d-4027-a30b-4eec092b0b1e-tmp\" (UniqueName: \"kubernetes.io/empty-dir/f619293a-1e5d-4027-a30b-4eec092b0b1e-tmp\") pod \"f619293a-1e5d-4027-a30b-4eec092b0b1e\" (UID: \"f619293a-1e5d-4027-a30b-4eec092b0b1e\") "
Oct 30 21:38:53.671727 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:53.671683    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f619293a-1e5d-4027-a30b-4eec092b0b1e-kube-api-access-582ld\" (UniqueName: \"kubernetes.io/projected/f619293a-1e5d-4027-a30b-4eec092b0b1e-kube-api-access-582ld\") pod \"f619293a-1e5d-4027-a30b-4eec092b0b1e\" (UID: \"f619293a-1e5d-4027-a30b-4eec092b0b1e\") "
Oct 30 21:38:53.671952 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:53.671729    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f619293a-1e5d-4027-a30b-4eec092b0b1e-console-config\" (UniqueName: \"kubernetes.io/configmap/f619293a-1e5d-4027-a30b-4eec092b0b1e-console-config\") pod \"f619293a-1e5d-4027-a30b-4eec092b0b1e\" (UID: \"f619293a-1e5d-4027-a30b-4eec092b0b1e\") "
Oct 30 21:38:53.671952 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:53.671755    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f619293a-1e5d-4027-a30b-4eec092b0b1e-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f619293a-1e5d-4027-a30b-4eec092b0b1e-console-oauth-config\") pod \"f619293a-1e5d-4027-a30b-4eec092b0b1e\" (UID: \"f619293a-1e5d-4027-a30b-4eec092b0b1e\") "
Oct 30 21:38:53.672070 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:53.672038    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f619293a-1e5d-4027-a30b-4eec092b0b1e-tmp" pod "f619293a-1e5d-4027-a30b-4eec092b0b1e" (UID: "f619293a-1e5d-4027-a30b-4eec092b0b1e"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:53.672182 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:53.672151    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f619293a-1e5d-4027-a30b-4eec092b0b1e-oauth-serving-cert" pod "f619293a-1e5d-4027-a30b-4eec092b0b1e" (UID: "f619293a-1e5d-4027-a30b-4eec092b0b1e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:38:53.672326 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:53.672302    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f619293a-1e5d-4027-a30b-4eec092b0b1e-trusted-ca-bundle" pod "f619293a-1e5d-4027-a30b-4eec092b0b1e" (UID: "f619293a-1e5d-4027-a30b-4eec092b0b1e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:38:53.672404 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:53.672366    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f619293a-1e5d-4027-a30b-4eec092b0b1e-service-ca" pod "f619293a-1e5d-4027-a30b-4eec092b0b1e" (UID: "f619293a-1e5d-4027-a30b-4eec092b0b1e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:38:53.672542 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:53.672516    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f619293a-1e5d-4027-a30b-4eec092b0b1e-console-config" pod "f619293a-1e5d-4027-a30b-4eec092b0b1e" (UID: "f619293a-1e5d-4027-a30b-4eec092b0b1e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:38:53.673998 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:53.673975    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f619293a-1e5d-4027-a30b-4eec092b0b1e-console-serving-cert" pod "f619293a-1e5d-4027-a30b-4eec092b0b1e" (UID: "f619293a-1e5d-4027-a30b-4eec092b0b1e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:38:53.674090 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:53.674029    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f619293a-1e5d-4027-a30b-4eec092b0b1e-console-oauth-config" pod "f619293a-1e5d-4027-a30b-4eec092b0b1e" (UID: "f619293a-1e5d-4027-a30b-4eec092b0b1e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:38:53.674090 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:53.674042    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f619293a-1e5d-4027-a30b-4eec092b0b1e-kube-api-access-582ld" pod "f619293a-1e5d-4027-a30b-4eec092b0b1e" (UID: "f619293a-1e5d-4027-a30b-4eec092b0b1e"). InnerVolumeSpecName "kube-api-access-582ld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:53.772631 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:53.772545    4871 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f619293a-1e5d-4027-a30b-4eec092b0b1e-service-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:38:53.772631 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:53.772579    4871 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f619293a-1e5d-4027-a30b-4eec092b0b1e-tmp\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:38:53.772631 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:53.772591    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-582ld\" (UniqueName: \"kubernetes.io/projected/f619293a-1e5d-4027-a30b-4eec092b0b1e-kube-api-access-582ld\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:38:53.772631 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:53.772601    4871 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f619293a-1e5d-4027-a30b-4eec092b0b1e-console-config\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:38:53.772631 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:53.772611    4871 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f619293a-1e5d-4027-a30b-4eec092b0b1e-console-oauth-config\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:38:53.772631 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:53.772619    4871 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f619293a-1e5d-4027-a30b-4eec092b0b1e-console-serving-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:38:53.772631 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:53.772628    4871 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f619293a-1e5d-4027-a30b-4eec092b0b1e-trusted-ca-bundle\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:38:53.772631 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:53.772637    4871 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f619293a-1e5d-4027-a30b-4eec092b0b1e-oauth-serving-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:38:54.348548 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:54.348509    4871 generic.go:356] "Generic (PLEG): container finished" podID="f619293a-1e5d-4027-a30b-4eec092b0b1e" containerID="7fbff780f4a214416f2d3a3127f003e9c45a01d1ac1e547d51ffb352bacb9ff7" exitCode=0
Oct 30 21:38:54.349002 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:54.348559    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9bb676567-hz5w8"
Oct 30 21:38:54.349002 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:54.348597    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9bb676567-hz5w8" event={"ID":"f619293a-1e5d-4027-a30b-4eec092b0b1e","Type":"ContainerDied","Data":"7fbff780f4a214416f2d3a3127f003e9c45a01d1ac1e547d51ffb352bacb9ff7"}
Oct 30 21:38:54.349002 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:54.348655    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9bb676567-hz5w8" event={"ID":"f619293a-1e5d-4027-a30b-4eec092b0b1e","Type":"ContainerDied","Data":"e2f6d63ceaa23fd668f13ae333e7dc0739a76ff36a1c469c551839d44954b4a4"}
Oct 30 21:38:54.349002 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:54.348679    4871 scope.go:122] "RemoveContainer" containerID="7fbff780f4a214416f2d3a3127f003e9c45a01d1ac1e547d51ffb352bacb9ff7"
Oct 30 21:38:54.359004 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:54.358981    4871 scope.go:122] "RemoveContainer" containerID="7fbff780f4a214416f2d3a3127f003e9c45a01d1ac1e547d51ffb352bacb9ff7"
Oct 30 21:38:54.359321 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:38:54.359300    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fbff780f4a214416f2d3a3127f003e9c45a01d1ac1e547d51ffb352bacb9ff7\": container with ID starting with 7fbff780f4a214416f2d3a3127f003e9c45a01d1ac1e547d51ffb352bacb9ff7 not found: ID does not exist" containerID="7fbff780f4a214416f2d3a3127f003e9c45a01d1ac1e547d51ffb352bacb9ff7"
Oct 30 21:38:54.359392 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:54.359329    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fbff780f4a214416f2d3a3127f003e9c45a01d1ac1e547d51ffb352bacb9ff7"} err="failed to get container status \"7fbff780f4a214416f2d3a3127f003e9c45a01d1ac1e547d51ffb352bacb9ff7\": rpc error: code = NotFound desc = could not find container \"7fbff780f4a214416f2d3a3127f003e9c45a01d1ac1e547d51ffb352bacb9ff7\": container with ID starting with 7fbff780f4a214416f2d3a3127f003e9c45a01d1ac1e547d51ffb352bacb9ff7 not found: ID does not exist"
Oct 30 21:38:54.508343 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:54.508305    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9bb676567-hz5w8"]
Oct 30 21:38:54.555408 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:54.555377    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-9bb676567-hz5w8"]
Oct 30 21:38:55.958232 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:38:55.958197    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f619293a-1e5d-4027-a30b-4eec092b0b1e" path="/var/lib/kubelet/pods/f619293a-1e5d-4027-a30b-4eec092b0b1e/volumes"
Oct 30 21:39:01.370096 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:39:01.370062    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:01.845088 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:39:01.845001    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:02.111921 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:39:02.111892    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:03.195858 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:39:03.195824    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:05.567921 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:39:05.567882    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:39:08.403785 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:39:08.403753    4871 generic.go:356] "Generic (PLEG): container finished" podID="4086d1bd-6e47-4d50-a7f9-5be23d123d6e" containerID="e01e645166be176533871d4d4a0a2e62a088ea447d78c75b822e28d17e7b8afd" exitCode=0
Oct 30 21:39:08.404176 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:39:08.403798    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-pf9px" event={"ID":"4086d1bd-6e47-4d50-a7f9-5be23d123d6e","Type":"ContainerDied","Data":"e01e645166be176533871d4d4a0a2e62a088ea447d78c75b822e28d17e7b8afd"}
Oct 30 21:39:08.404176 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:39:08.404088    4871 scope.go:122] "RemoveContainer" containerID="e01e645166be176533871d4d4a0a2e62a088ea447d78c75b822e28d17e7b8afd"
Oct 30 21:39:09.413310 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:39:09.413267    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-pf9px" event={"ID":"4086d1bd-6e47-4d50-a7f9-5be23d123d6e","Type":"ContainerStarted","Data":"32ddf369cde7ccdf8d7d615ece4c866393b980047db4463353fbbef441c2fe23"}
Oct 30 21:39:14.434279 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:39:14.434217    4871 generic.go:356] "Generic (PLEG): container finished" podID="fb70ad86-4b45-4a53-a3cb-1f6b59640708" containerID="314e80b5158e2c5213dc954620b41d97abef4c7368175cb60a52ab81231ed958" exitCode=0
Oct 30 21:39:14.434663 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:39:14.434287    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-qqxdq" event={"ID":"fb70ad86-4b45-4a53-a3cb-1f6b59640708","Type":"ContainerDied","Data":"314e80b5158e2c5213dc954620b41d97abef4c7368175cb60a52ab81231ed958"}
Oct 30 21:39:14.434663 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:39:14.434606    4871 scope.go:122] "RemoveContainer" containerID="314e80b5158e2c5213dc954620b41d97abef4c7368175cb60a52ab81231ed958"
Oct 30 21:39:15.439877 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:39:15.439838    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-qqxdq" event={"ID":"fb70ad86-4b45-4a53-a3cb-1f6b59640708","Type":"ContainerStarted","Data":"c35d77d8ff155ef21a2698942e768f791ea4d6fd9bff2355b1061b97e92d4f72"}
Oct 30 21:39:23.942363 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:39:23.942326    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:24.249134 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:39:24.249053    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:25.334556 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:39:25.334520    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:28.268648 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:39:28.268615    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:35.567491 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:39:35.567455    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:40:05.566993 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:40:05.566903    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:40:28.630403 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:28.630368    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:35.565841 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:40:35.565807    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:40:46.126733 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:46.126684    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:46.433099 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:46.433014    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:48.184254 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:48.184207    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:48.277025 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:48.276990    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ndg4g"]
Oct 30 21:40:48.277294 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:48.277281    4871 cpu_manager.go:414] "containerMap: removing container" podUID="0dca73b7-6526-48ac-a76e-70e441debc00" containerName="registry"
Oct 30 21:40:48.277342 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:48.277296    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0dca73b7-6526-48ac-a76e-70e441debc00" containerName="registry"
Oct 30 21:40:48.277342 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:48.277320    4871 cpu_manager.go:414] "containerMap: removing container" podUID="30931e26-479e-4ddb-8ce3-9b2b52dc0e8e" containerName="registry"
Oct 30 21:40:48.277342 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:48.277325    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="30931e26-479e-4ddb-8ce3-9b2b52dc0e8e" containerName="registry"
Oct 30 21:40:48.277342 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:48.277340    4871 cpu_manager.go:414] "containerMap: removing container" podUID="f619293a-1e5d-4027-a30b-4eec092b0b1e" containerName="console"
Oct 30 21:40:48.277468 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:48.277345    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f619293a-1e5d-4027-a30b-4eec092b0b1e" containerName="console"
Oct 30 21:40:48.277468 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:48.277395    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="30931e26-479e-4ddb-8ce3-9b2b52dc0e8e" containerName="registry"
Oct 30 21:40:48.277468 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:48.277404    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="f619293a-1e5d-4027-a30b-4eec092b0b1e" containerName="console"
Oct 30 21:40:48.277468 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:48.277410    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="0dca73b7-6526-48ac-a76e-70e441debc00" containerName="registry"
Oct 30 21:40:48.280009 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:48.279993    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ndg4g"
Oct 30 21:40:48.376623 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:48.376589    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 21:40:48.376623 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:48.376612    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e644421527c169d876fdb59235c927986d6bd2-9d2815fc-dockercfg-qgv9p\""
Oct 30 21:40:48.376623 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:48.376589    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 21:40:48.377618 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:48.377600    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ndg4g"]
Oct 30 21:40:48.384905 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:48.384884    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:48.424731 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:48.424694    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/56a9d68e-fdea-4bd4-a96c-855199901791-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ndg4g\" (UID: \"56a9d68e-fdea-4bd4-a96c-855199901791\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ndg4g"
Oct 30 21:40:48.424896 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:48.424738    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/56a9d68e-fdea-4bd4-a96c-855199901791-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ndg4g\" (UID: \"56a9d68e-fdea-4bd4-a96c-855199901791\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ndg4g"
Oct 30 21:40:48.424896 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:48.424803    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfxlq\" (UniqueName: \"kubernetes.io/projected/56a9d68e-fdea-4bd4-a96c-855199901791-kube-api-access-kfxlq\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ndg4g\" (UID: \"56a9d68e-fdea-4bd4-a96c-855199901791\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ndg4g"
Oct 30 21:40:48.525254 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:48.525155    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/56a9d68e-fdea-4bd4-a96c-855199901791-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ndg4g\" (UID: \"56a9d68e-fdea-4bd4-a96c-855199901791\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ndg4g"
Oct 30 21:40:48.525254 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:48.525201    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/56a9d68e-fdea-4bd4-a96c-855199901791-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ndg4g\" (UID: \"56a9d68e-fdea-4bd4-a96c-855199901791\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ndg4g"
Oct 30 21:40:48.525446 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:48.525263    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfxlq\" (UniqueName: \"kubernetes.io/projected/56a9d68e-fdea-4bd4-a96c-855199901791-kube-api-access-kfxlq\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ndg4g\" (UID: \"56a9d68e-fdea-4bd4-a96c-855199901791\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ndg4g"
Oct 30 21:40:48.525602 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:48.525582    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/56a9d68e-fdea-4bd4-a96c-855199901791-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ndg4g\" (UID: \"56a9d68e-fdea-4bd4-a96c-855199901791\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ndg4g"
Oct 30 21:40:48.525664 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:48.525643    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/56a9d68e-fdea-4bd4-a96c-855199901791-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ndg4g\" (UID: \"56a9d68e-fdea-4bd4-a96c-855199901791\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ndg4g"
Oct 30 21:40:48.591355 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:48.591323    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfxlq\" (UniqueName: \"kubernetes.io/projected/56a9d68e-fdea-4bd4-a96c-855199901791-kube-api-access-kfxlq\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ndg4g\" (UID: \"56a9d68e-fdea-4bd4-a96c-855199901791\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ndg4g"
Oct 30 21:40:48.594162 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:48.594140    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ndg4g"
Oct 30 21:40:48.957603 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:48.957568    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ndg4g"]
Oct 30 21:40:48.960883 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:40:48.960854    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56a9d68e_fdea_4bd4_a96c_855199901791.slice/crio-1f4af9a67f6cc9725acb2d1d98c6cabc38f20fdd8cadefc7fb47b106086f3bcf WatchSource:0}: Error finding container 1f4af9a67f6cc9725acb2d1d98c6cabc38f20fdd8cadefc7fb47b106086f3bcf: Status 404 returned error can't find the container with id 1f4af9a67f6cc9725acb2d1d98c6cabc38f20fdd8cadefc7fb47b106086f3bcf
Oct 30 21:40:49.132739 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:49.132701    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqcvcw"]
Oct 30 21:40:49.136937 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:49.136911    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqcvcw"
Oct 30 21:40:49.188589 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:49.188549    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c7537007944577f64df34960ace3c69136c416-139ec370-dockercfg-bf8pt\""
Oct 30 21:40:49.192859 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:49.192826    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqcvcw"]
Oct 30 21:40:49.230681 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:49.230589    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/570a9217-1578-43e4-a5fd-65c8b261b535-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqcvcw\" (UID: \"570a9217-1578-43e4-a5fd-65c8b261b535\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqcvcw"
Oct 30 21:40:49.230681 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:49.230637    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/570a9217-1578-43e4-a5fd-65c8b261b535-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqcvcw\" (UID: \"570a9217-1578-43e4-a5fd-65c8b261b535\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqcvcw"
Oct 30 21:40:49.230681 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:49.230671    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcglm\" (UniqueName: \"kubernetes.io/projected/570a9217-1578-43e4-a5fd-65c8b261b535-kube-api-access-kcglm\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqcvcw\" (UID: \"570a9217-1578-43e4-a5fd-65c8b261b535\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqcvcw"
Oct 30 21:40:49.331154 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:49.331114    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/570a9217-1578-43e4-a5fd-65c8b261b535-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqcvcw\" (UID: \"570a9217-1578-43e4-a5fd-65c8b261b535\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqcvcw"
Oct 30 21:40:49.331362 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:49.331176    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/570a9217-1578-43e4-a5fd-65c8b261b535-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqcvcw\" (UID: \"570a9217-1578-43e4-a5fd-65c8b261b535\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqcvcw"
Oct 30 21:40:49.331362 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:49.331217    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcglm\" (UniqueName: \"kubernetes.io/projected/570a9217-1578-43e4-a5fd-65c8b261b535-kube-api-access-kcglm\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqcvcw\" (UID: \"570a9217-1578-43e4-a5fd-65c8b261b535\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqcvcw"
Oct 30 21:40:49.331572 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:49.331546    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/570a9217-1578-43e4-a5fd-65c8b261b535-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqcvcw\" (UID: \"570a9217-1578-43e4-a5fd-65c8b261b535\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqcvcw"
Oct 30 21:40:49.331607 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:49.331556    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/570a9217-1578-43e4-a5fd-65c8b261b535-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqcvcw\" (UID: \"570a9217-1578-43e4-a5fd-65c8b261b535\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqcvcw"
Oct 30 21:40:49.389449 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:49.389422    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcglm\" (UniqueName: \"kubernetes.io/projected/570a9217-1578-43e4-a5fd-65c8b261b535-kube-api-access-kcglm\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqcvcw\" (UID: \"570a9217-1578-43e4-a5fd-65c8b261b535\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqcvcw"
Oct 30 21:40:49.452620 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:49.452583    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqcvcw"
Oct 30 21:40:49.739539 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:49.739508    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:49.774001 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:49.773904    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ndg4g" event={"ID":"56a9d68e-fdea-4bd4-a96c-855199901791","Type":"ContainerStarted","Data":"1f4af9a67f6cc9725acb2d1d98c6cabc38f20fdd8cadefc7fb47b106086f3bcf"}
Oct 30 21:40:49.814855 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:49.814825    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqcvcw"]
Oct 30 21:40:49.818181 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:40:49.818146    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod570a9217_1578_43e4_a5fd_65c8b261b535.slice/crio-fd1f8406ac78d06300f424b674eae0037a3ae931d8114f2c0aa9bd8753147e78 WatchSource:0}: Error finding container fd1f8406ac78d06300f424b674eae0037a3ae931d8114f2c0aa9bd8753147e78: Status 404 returned error can't find the container with id fd1f8406ac78d06300f424b674eae0037a3ae931d8114f2c0aa9bd8753147e78
Oct 30 21:40:49.836607 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:49.836581    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6b9qt6"]
Oct 30 21:40:49.840898 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:49.840882    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6b9qt6"
Oct 30 21:40:49.890440 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:49.890404    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3-b83c8cba-dockercfg-4r8sh\""
Oct 30 21:40:49.923308 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:49.923271    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6b9qt6"]
Oct 30 21:40:49.935019 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:49.934993    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/51e4633e-d58b-4843-92e1-35c0bfb629d3-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6b9qt6\" (UID: \"51e4633e-d58b-4843-92e1-35c0bfb629d3\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6b9qt6"
Oct 30 21:40:49.935180 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:49.935113    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cqks\" (UniqueName: \"kubernetes.io/projected/51e4633e-d58b-4843-92e1-35c0bfb629d3-kube-api-access-9cqks\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6b9qt6\" (UID: \"51e4633e-d58b-4843-92e1-35c0bfb629d3\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6b9qt6"
Oct 30 21:40:49.935233 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:49.935176    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/51e4633e-d58b-4843-92e1-35c0bfb629d3-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6b9qt6\" (UID: \"51e4633e-d58b-4843-92e1-35c0bfb629d3\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6b9qt6"
Oct 30 21:40:49.936488 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:49.936464    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqplt6"]
Oct 30 21:40:49.939613 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:49.939594    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vsdmv"]
Oct 30 21:40:49.939736 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:49.939717    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqplt6"
Oct 30 21:40:49.942659 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:49.942639    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vsdmv"
Oct 30 21:40:49.990417 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:49.990384    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e3d011e69d9573cabb8c359e5ac709eb5ac240-be78d160-dockercfg-dxsh6\""
Oct 30 21:40:49.997086 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:49.997057    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c4959aeb960c8e20c88ac2de924a35b8221494-145dcd5a-dockercfg-nqwjk\""
Oct 30 21:40:49.997605 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:49.997583    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqplt6"]
Oct 30 21:40:50.033514 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.033412    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vsdmv"]
Oct 30 21:40:50.036453 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.036422    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/51e4633e-d58b-4843-92e1-35c0bfb629d3-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6b9qt6\" (UID: \"51e4633e-d58b-4843-92e1-35c0bfb629d3\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6b9qt6"
Oct 30 21:40:50.036596 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.036476    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/689c9fec-d928-483c-aea2-d5cd577213ff-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqplt6\" (UID: \"689c9fec-d928-483c-aea2-d5cd577213ff\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqplt6"
Oct 30 21:40:50.036596 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.036504    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/689c9fec-d928-483c-aea2-d5cd577213ff-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqplt6\" (UID: \"689c9fec-d928-483c-aea2-d5cd577213ff\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqplt6"
Oct 30 21:40:50.036905 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.036866    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/53353129-6959-43a8-83c5-e845c7869516-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vsdmv\" (UID: \"53353129-6959-43a8-83c5-e845c7869516\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vsdmv"
Oct 30 21:40:50.037132 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.037101    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cqks\" (UniqueName: \"kubernetes.io/projected/51e4633e-d58b-4843-92e1-35c0bfb629d3-kube-api-access-9cqks\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6b9qt6\" (UID: \"51e4633e-d58b-4843-92e1-35c0bfb629d3\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6b9qt6"
Oct 30 21:40:50.037225 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.037142    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndv95\" (UniqueName: \"kubernetes.io/projected/53353129-6959-43a8-83c5-e845c7869516-kube-api-access-ndv95\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vsdmv\" (UID: \"53353129-6959-43a8-83c5-e845c7869516\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vsdmv"
Oct 30 21:40:50.037225 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.037201    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/51e4633e-d58b-4843-92e1-35c0bfb629d3-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6b9qt6\" (UID: \"51e4633e-d58b-4843-92e1-35c0bfb629d3\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6b9qt6"
Oct 30 21:40:50.037339 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.037233    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxmbn\" (UniqueName: \"kubernetes.io/projected/689c9fec-d928-483c-aea2-d5cd577213ff-kube-api-access-zxmbn\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqplt6\" (UID: \"689c9fec-d928-483c-aea2-d5cd577213ff\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqplt6"
Oct 30 21:40:50.037488 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.037450    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/53353129-6959-43a8-83c5-e845c7869516-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vsdmv\" (UID: \"53353129-6959-43a8-83c5-e845c7869516\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vsdmv"
Oct 30 21:40:50.038257 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.038149    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/51e4633e-d58b-4843-92e1-35c0bfb629d3-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6b9qt6\" (UID: \"51e4633e-d58b-4843-92e1-35c0bfb629d3\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6b9qt6"
Oct 30 21:40:50.038257 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.038195    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/51e4633e-d58b-4843-92e1-35c0bfb629d3-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6b9qt6\" (UID: \"51e4633e-d58b-4843-92e1-35c0bfb629d3\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6b9qt6"
Oct 30 21:40:50.039354 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.039336    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fjfzw"]
Oct 30 21:40:50.042848 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.042829    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fjfzw"
Oct 30 21:40:50.099969 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.099935    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"da112bd609bff87f7bacd2ba941225d7cff15c-bd73b2fc-dockercfg-j77h8\""
Oct 30 21:40:50.108960 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.108927    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cqks\" (UniqueName: \"kubernetes.io/projected/51e4633e-d58b-4843-92e1-35c0bfb629d3-kube-api-access-9cqks\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6b9qt6\" (UID: \"51e4633e-d58b-4843-92e1-35c0bfb629d3\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6b9qt6"
Oct 30 21:40:50.129529 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.129477    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fjfzw"]
Oct 30 21:40:50.138047 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.138010    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/036a0620-585b-4e3f-bb7e-5f4bb7799498-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fjfzw\" (UID: \"036a0620-585b-4e3f-bb7e-5f4bb7799498\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fjfzw"
Oct 30 21:40:50.138214 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.138059    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zc5cx\" (UniqueName: \"kubernetes.io/projected/036a0620-585b-4e3f-bb7e-5f4bb7799498-kube-api-access-zc5cx\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fjfzw\" (UID: \"036a0620-585b-4e3f-bb7e-5f4bb7799498\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fjfzw"
Oct 30 21:40:50.138214 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.138098    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxmbn\" (UniqueName: \"kubernetes.io/projected/689c9fec-d928-483c-aea2-d5cd577213ff-kube-api-access-zxmbn\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqplt6\" (UID: \"689c9fec-d928-483c-aea2-d5cd577213ff\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqplt6"
Oct 30 21:40:50.138214 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.138186    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/53353129-6959-43a8-83c5-e845c7869516-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vsdmv\" (UID: \"53353129-6959-43a8-83c5-e845c7869516\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vsdmv"
Oct 30 21:40:50.138420 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.138334    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/689c9fec-d928-483c-aea2-d5cd577213ff-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqplt6\" (UID: \"689c9fec-d928-483c-aea2-d5cd577213ff\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqplt6"
Oct 30 21:40:50.138420 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.138366    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/689c9fec-d928-483c-aea2-d5cd577213ff-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqplt6\" (UID: \"689c9fec-d928-483c-aea2-d5cd577213ff\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqplt6"
Oct 30 21:40:50.138420 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.138389    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/036a0620-585b-4e3f-bb7e-5f4bb7799498-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fjfzw\" (UID: \"036a0620-585b-4e3f-bb7e-5f4bb7799498\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fjfzw"
Oct 30 21:40:50.138583 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.138465    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/53353129-6959-43a8-83c5-e845c7869516-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vsdmv\" (UID: \"53353129-6959-43a8-83c5-e845c7869516\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vsdmv"
Oct 30 21:40:50.138583 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.138496    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndv95\" (UniqueName: \"kubernetes.io/projected/53353129-6959-43a8-83c5-e845c7869516-kube-api-access-ndv95\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vsdmv\" (UID: \"53353129-6959-43a8-83c5-e845c7869516\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vsdmv"
Oct 30 21:40:50.138716 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.138685    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/53353129-6959-43a8-83c5-e845c7869516-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vsdmv\" (UID: \"53353129-6959-43a8-83c5-e845c7869516\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vsdmv"
Oct 30 21:40:50.138770 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.138744    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/689c9fec-d928-483c-aea2-d5cd577213ff-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqplt6\" (UID: \"689c9fec-d928-483c-aea2-d5cd577213ff\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqplt6"
Oct 30 21:40:50.138833 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.138784    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/689c9fec-d928-483c-aea2-d5cd577213ff-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqplt6\" (UID: \"689c9fec-d928-483c-aea2-d5cd577213ff\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqplt6"
Oct 30 21:40:50.138929 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.138910    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/53353129-6959-43a8-83c5-e845c7869516-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vsdmv\" (UID: \"53353129-6959-43a8-83c5-e845c7869516\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vsdmv"
Oct 30 21:40:50.155973 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.155944    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6b9qt6"
Oct 30 21:40:50.201653 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.201616    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxmbn\" (UniqueName: \"kubernetes.io/projected/689c9fec-d928-483c-aea2-d5cd577213ff-kube-api-access-zxmbn\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqplt6\" (UID: \"689c9fec-d928-483c-aea2-d5cd577213ff\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqplt6"
Oct 30 21:40:50.203332 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.203307    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndv95\" (UniqueName: \"kubernetes.io/projected/53353129-6959-43a8-83c5-e845c7869516-kube-api-access-ndv95\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vsdmv\" (UID: \"53353129-6959-43a8-83c5-e845c7869516\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vsdmv"
Oct 30 21:40:50.208776 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.208750    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agzz78"]
Oct 30 21:40:50.230780 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.230747    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agzz78"
Oct 30 21:40:50.239074 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.239039    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/036a0620-585b-4e3f-bb7e-5f4bb7799498-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fjfzw\" (UID: \"036a0620-585b-4e3f-bb7e-5f4bb7799498\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fjfzw"
Oct 30 21:40:50.239275 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.239088    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zc5cx\" (UniqueName: \"kubernetes.io/projected/036a0620-585b-4e3f-bb7e-5f4bb7799498-kube-api-access-zc5cx\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fjfzw\" (UID: \"036a0620-585b-4e3f-bb7e-5f4bb7799498\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fjfzw"
Oct 30 21:40:50.239275 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.239153    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/036a0620-585b-4e3f-bb7e-5f4bb7799498-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fjfzw\" (UID: \"036a0620-585b-4e3f-bb7e-5f4bb7799498\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fjfzw"
Oct 30 21:40:50.239514 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.239451    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/036a0620-585b-4e3f-bb7e-5f4bb7799498-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fjfzw\" (UID: \"036a0620-585b-4e3f-bb7e-5f4bb7799498\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fjfzw"
Oct 30 21:40:50.240302 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.240280    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/036a0620-585b-4e3f-bb7e-5f4bb7799498-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fjfzw\" (UID: \"036a0620-585b-4e3f-bb7e-5f4bb7799498\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fjfzw"
Oct 30 21:40:50.256034 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.256006    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqplt6"
Oct 30 21:40:50.263001 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.262969    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vsdmv"
Oct 30 21:40:50.270088 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.270064    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agzz78"]
Oct 30 21:40:50.283705 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.283626    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"6541d93aaee335d6227a8778ab4e840bbe7f52-f98fbcf8-dockercfg-mrdh8\""
Oct 30 21:40:50.300114 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.298809    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zc5cx\" (UniqueName: \"kubernetes.io/projected/036a0620-585b-4e3f-bb7e-5f4bb7799498-kube-api-access-zc5cx\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fjfzw\" (UID: \"036a0620-585b-4e3f-bb7e-5f4bb7799498\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fjfzw"
Oct 30 21:40:50.340437 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.340157    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dxxf\" (UniqueName: \"kubernetes.io/projected/cfdfd8e2-c269-41a5-847f-2760cd6577c7-kube-api-access-2dxxf\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agzz78\" (UID: \"cfdfd8e2-c269-41a5-847f-2760cd6577c7\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agzz78"
Oct 30 21:40:50.340437 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.340228    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cfdfd8e2-c269-41a5-847f-2760cd6577c7-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agzz78\" (UID: \"cfdfd8e2-c269-41a5-847f-2760cd6577c7\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agzz78"
Oct 30 21:40:50.340437 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.340314    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cfdfd8e2-c269-41a5-847f-2760cd6577c7-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agzz78\" (UID: \"cfdfd8e2-c269-41a5-847f-2760cd6577c7\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agzz78"
Oct 30 21:40:50.359655 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.359622    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fjfzw"
Oct 30 21:40:50.440837 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.440802    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cfdfd8e2-c269-41a5-847f-2760cd6577c7-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agzz78\" (UID: \"cfdfd8e2-c269-41a5-847f-2760cd6577c7\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agzz78"
Oct 30 21:40:50.440962 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.440860    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dxxf\" (UniqueName: \"kubernetes.io/projected/cfdfd8e2-c269-41a5-847f-2760cd6577c7-kube-api-access-2dxxf\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agzz78\" (UID: \"cfdfd8e2-c269-41a5-847f-2760cd6577c7\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agzz78"
Oct 30 21:40:50.440962 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.440942    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cfdfd8e2-c269-41a5-847f-2760cd6577c7-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agzz78\" (UID: \"cfdfd8e2-c269-41a5-847f-2760cd6577c7\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agzz78"
Oct 30 21:40:50.441429 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.441407    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cfdfd8e2-c269-41a5-847f-2760cd6577c7-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agzz78\" (UID: \"cfdfd8e2-c269-41a5-847f-2760cd6577c7\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agzz78"
Oct 30 21:40:50.441693 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.441676    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cfdfd8e2-c269-41a5-847f-2760cd6577c7-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agzz78\" (UID: \"cfdfd8e2-c269-41a5-847f-2760cd6577c7\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agzz78"
Oct 30 21:40:50.500354 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.500323    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dxxf\" (UniqueName: \"kubernetes.io/projected/cfdfd8e2-c269-41a5-847f-2760cd6577c7-kube-api-access-2dxxf\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agzz78\" (UID: \"cfdfd8e2-c269-41a5-847f-2760cd6577c7\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agzz78"
Oct 30 21:40:50.528730 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.528705    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6b9qt6"]
Oct 30 21:40:50.531596 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:40:50.531559    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51e4633e_d58b_4843_92e1_35c0bfb629d3.slice/crio-0c8817db71004561adddd47366b67fadb44c367b344adf13aae8c87bf62ee50d WatchSource:0}: Error finding container 0c8817db71004561adddd47366b67fadb44c367b344adf13aae8c87bf62ee50d: Status 404 returned error can't find the container with id 0c8817db71004561adddd47366b67fadb44c367b344adf13aae8c87bf62ee50d
Oct 30 21:40:50.551061 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.551028    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agzz78"
Oct 30 21:40:50.630581 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.630547    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqplt6"]
Oct 30 21:40:50.631434 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:40:50.631407    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod689c9fec_d928_483c_aea2_d5cd577213ff.slice/crio-fc1e0f562eef1f0156656a4aace70dfc49c46e6d043feb01423abe03437e5180 WatchSource:0}: Error finding container fc1e0f562eef1f0156656a4aace70dfc49c46e6d043feb01423abe03437e5180: Status 404 returned error can't find the container with id fc1e0f562eef1f0156656a4aace70dfc49c46e6d043feb01423abe03437e5180
Oct 30 21:40:50.636490 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.636462    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vsdmv"]
Oct 30 21:40:50.641123 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:40:50.641079    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53353129_6959_43a8_83c5_e845c7869516.slice/crio-5bd7bc6e7d0f739a857e2bd3cd2bf1182dd947db6c27bfb4861d26d187476407 WatchSource:0}: Error finding container 5bd7bc6e7d0f739a857e2bd3cd2bf1182dd947db6c27bfb4861d26d187476407: Status 404 returned error can't find the container with id 5bd7bc6e7d0f739a857e2bd3cd2bf1182dd947db6c27bfb4861d26d187476407
Oct 30 21:40:50.741441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.741408    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fjfzw"]
Oct 30 21:40:50.745965 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:40:50.745926    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod036a0620_585b_4e3f_bb7e_5f4bb7799498.slice/crio-7c42e83147ad4f90a4c7599fab9979b2117e26dbc729cc78439c18f0dbe9af37 WatchSource:0}: Error finding container 7c42e83147ad4f90a4c7599fab9979b2117e26dbc729cc78439c18f0dbe9af37: Status 404 returned error can't find the container with id 7c42e83147ad4f90a4c7599fab9979b2117e26dbc729cc78439c18f0dbe9af37
Oct 30 21:40:50.785848 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.785765    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6b9qt6" event={"ID":"51e4633e-d58b-4843-92e1-35c0bfb629d3","Type":"ContainerStarted","Data":"0c8817db71004561adddd47366b67fadb44c367b344adf13aae8c87bf62ee50d"}
Oct 30 21:40:50.787279 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.787224    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqcvcw" event={"ID":"570a9217-1578-43e4-a5fd-65c8b261b535","Type":"ContainerStarted","Data":"fd1f8406ac78d06300f424b674eae0037a3ae931d8114f2c0aa9bd8753147e78"}
Oct 30 21:40:50.788455 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.788431    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fjfzw" event={"ID":"036a0620-585b-4e3f-bb7e-5f4bb7799498","Type":"ContainerStarted","Data":"7c42e83147ad4f90a4c7599fab9979b2117e26dbc729cc78439c18f0dbe9af37"}
Oct 30 21:40:50.789845 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.789817    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqplt6" event={"ID":"689c9fec-d928-483c-aea2-d5cd577213ff","Type":"ContainerStarted","Data":"fc1e0f562eef1f0156656a4aace70dfc49c46e6d043feb01423abe03437e5180"}
Oct 30 21:40:50.791062 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.791038    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vsdmv" event={"ID":"53353129-6959-43a8-83c5-e845c7869516","Type":"ContainerStarted","Data":"5bd7bc6e7d0f739a857e2bd3cd2bf1182dd947db6c27bfb4861d26d187476407"}
Oct 30 21:40:50.929005 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:50.928974    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agzz78"]
Oct 30 21:40:50.930925 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:40:50.930900    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfdfd8e2_c269_41a5_847f_2760cd6577c7.slice/crio-ed43f16e0318efeb525011bfd73652a581961a9dfd32088049eeb40a5e729fde WatchSource:0}: Error finding container ed43f16e0318efeb525011bfd73652a581961a9dfd32088049eeb40a5e729fde: Status 404 returned error can't find the container with id ed43f16e0318efeb525011bfd73652a581961a9dfd32088049eeb40a5e729fde
Oct 30 21:40:51.800247 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:51.800140    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agzz78" event={"ID":"cfdfd8e2-c269-41a5-847f-2760cd6577c7","Type":"ContainerStarted","Data":"ed43f16e0318efeb525011bfd73652a581961a9dfd32088049eeb40a5e729fde"}
Oct 30 21:40:53.729226 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:53.729191    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx"]
Oct 30 21:40:53.732752 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:53.732726    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx"
Oct 30 21:40:53.831762 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:53.831576    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx"]
Oct 30 21:40:53.871573 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:53.871205    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8d401654-8a4c-4a79-b2d3-612765312421-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx\" (UID: \"8d401654-8a4c-4a79-b2d3-612765312421\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx"
Oct 30 21:40:53.871573 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:53.871337    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8d401654-8a4c-4a79-b2d3-612765312421-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx\" (UID: \"8d401654-8a4c-4a79-b2d3-612765312421\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx"
Oct 30 21:40:53.871573 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:53.871365    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fn4k7\" (UniqueName: \"kubernetes.io/projected/8d401654-8a4c-4a79-b2d3-612765312421-kube-api-access-fn4k7\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx\" (UID: \"8d401654-8a4c-4a79-b2d3-612765312421\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx"
Oct 30 21:40:53.971851 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:53.971816    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8d401654-8a4c-4a79-b2d3-612765312421-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx\" (UID: \"8d401654-8a4c-4a79-b2d3-612765312421\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx"
Oct 30 21:40:53.971851 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:53.971850    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fn4k7\" (UniqueName: \"kubernetes.io/projected/8d401654-8a4c-4a79-b2d3-612765312421-kube-api-access-fn4k7\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx\" (UID: \"8d401654-8a4c-4a79-b2d3-612765312421\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx"
Oct 30 21:40:53.972093 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:53.971896    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8d401654-8a4c-4a79-b2d3-612765312421-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx\" (UID: \"8d401654-8a4c-4a79-b2d3-612765312421\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx"
Oct 30 21:40:53.972256 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:53.972225    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8d401654-8a4c-4a79-b2d3-612765312421-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx\" (UID: \"8d401654-8a4c-4a79-b2d3-612765312421\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx"
Oct 30 21:40:53.972499 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:53.972479    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8d401654-8a4c-4a79-b2d3-612765312421-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx\" (UID: \"8d401654-8a4c-4a79-b2d3-612765312421\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx"
Oct 30 21:40:54.054132 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:54.054056    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fn4k7\" (UniqueName: \"kubernetes.io/projected/8d401654-8a4c-4a79-b2d3-612765312421-kube-api-access-fn4k7\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx\" (UID: \"8d401654-8a4c-4a79-b2d3-612765312421\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx"
Oct 30 21:40:54.054300 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:54.054249    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx"
Oct 30 21:40:54.110198 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:54.110102    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:54.298074 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:54.298030    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:54.590788 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:54.590726    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx"]
Oct 30 21:40:54.600820 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:40:54.600785    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d401654_8a4c_4a79_b2d3_612765312421.slice/crio-aea3b7da9b7840e0c6788cb479ce0082c9d8374ce3e8fbc42667430da4c1800f WatchSource:0}: Error finding container aea3b7da9b7840e0c6788cb479ce0082c9d8374ce3e8fbc42667430da4c1800f: Status 404 returned error can't find the container with id aea3b7da9b7840e0c6788cb479ce0082c9d8374ce3e8fbc42667430da4c1800f
Oct 30 21:40:54.837990 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:54.837950    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx" event={"ID":"8d401654-8a4c-4a79-b2d3-612765312421","Type":"ContainerStarted","Data":"aea3b7da9b7840e0c6788cb479ce0082c9d8374ce3e8fbc42667430da4c1800f"}
Oct 30 21:40:55.677429 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:55.677389    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"]
Oct 30 21:40:55.682172 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:55.681300    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"
Oct 30 21:40:55.731938 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:55.731905    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-dvk6k\""
Oct 30 21:40:55.753548 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:55.753519    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 30 21:40:55.780063 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:55.779919    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"]
Oct 30 21:40:55.792053 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:55.792012    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c5a0598-58b9-4032-8e23-27daf0444149-istio-envoy\") pod \"maas-default-gateway-openshift-default-7df854bf67-hkg2v\" (UID: \"2c5a0598-58b9-4032-8e23-27daf0444149\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"
Oct 30 21:40:55.792230 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:55.792066    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c5a0598-58b9-4032-8e23-27daf0444149-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7df854bf67-hkg2v\" (UID: \"2c5a0598-58b9-4032-8e23-27daf0444149\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"
Oct 30 21:40:55.792230 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:55.792099    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c5a0598-58b9-4032-8e23-27daf0444149-workload-certs\") pod \"maas-default-gateway-openshift-default-7df854bf67-hkg2v\" (UID: \"2c5a0598-58b9-4032-8e23-27daf0444149\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"
Oct 30 21:40:55.792230 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:55.792128    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c5a0598-58b9-4032-8e23-27daf0444149-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7df854bf67-hkg2v\" (UID: \"2c5a0598-58b9-4032-8e23-27daf0444149\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"
Oct 30 21:40:55.792230 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:55.792151    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c5a0598-58b9-4032-8e23-27daf0444149-workload-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-hkg2v\" (UID: \"2c5a0598-58b9-4032-8e23-27daf0444149\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"
Oct 30 21:40:55.792516 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:55.792491    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c5a0598-58b9-4032-8e23-27daf0444149-istio-data\") pod \"maas-default-gateway-openshift-default-7df854bf67-hkg2v\" (UID: \"2c5a0598-58b9-4032-8e23-27daf0444149\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"
Oct 30 21:40:55.792614 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:55.792561    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c5a0598-58b9-4032-8e23-27daf0444149-istio-token\") pod \"maas-default-gateway-openshift-default-7df854bf67-hkg2v\" (UID: \"2c5a0598-58b9-4032-8e23-27daf0444149\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"
Oct 30 21:40:55.792614 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:55.792592    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c5a0598-58b9-4032-8e23-27daf0444149-credential-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-hkg2v\" (UID: \"2c5a0598-58b9-4032-8e23-27daf0444149\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"
Oct 30 21:40:55.792614 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:55.792620    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nn9gc\" (UniqueName: \"kubernetes.io/projected/2c5a0598-58b9-4032-8e23-27daf0444149-kube-api-access-nn9gc\") pod \"maas-default-gateway-openshift-default-7df854bf67-hkg2v\" (UID: \"2c5a0598-58b9-4032-8e23-27daf0444149\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"
Oct 30 21:40:55.793624 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:55.793314    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:55.895714 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:55.893345    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c5a0598-58b9-4032-8e23-27daf0444149-istio-token\") pod \"maas-default-gateway-openshift-default-7df854bf67-hkg2v\" (UID: \"2c5a0598-58b9-4032-8e23-27daf0444149\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"
Oct 30 21:40:55.895714 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:55.893389    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c5a0598-58b9-4032-8e23-27daf0444149-credential-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-hkg2v\" (UID: \"2c5a0598-58b9-4032-8e23-27daf0444149\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"
Oct 30 21:40:55.895714 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:55.893407    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nn9gc\" (UniqueName: \"kubernetes.io/projected/2c5a0598-58b9-4032-8e23-27daf0444149-kube-api-access-nn9gc\") pod \"maas-default-gateway-openshift-default-7df854bf67-hkg2v\" (UID: \"2c5a0598-58b9-4032-8e23-27daf0444149\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"
Oct 30 21:40:55.895714 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:55.893441    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c5a0598-58b9-4032-8e23-27daf0444149-istio-envoy\") pod \"maas-default-gateway-openshift-default-7df854bf67-hkg2v\" (UID: \"2c5a0598-58b9-4032-8e23-27daf0444149\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"
Oct 30 21:40:55.895714 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:55.893481    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c5a0598-58b9-4032-8e23-27daf0444149-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7df854bf67-hkg2v\" (UID: \"2c5a0598-58b9-4032-8e23-27daf0444149\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"
Oct 30 21:40:55.895714 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:55.893527    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c5a0598-58b9-4032-8e23-27daf0444149-workload-certs\") pod \"maas-default-gateway-openshift-default-7df854bf67-hkg2v\" (UID: \"2c5a0598-58b9-4032-8e23-27daf0444149\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"
Oct 30 21:40:55.895714 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:55.893552    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c5a0598-58b9-4032-8e23-27daf0444149-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7df854bf67-hkg2v\" (UID: \"2c5a0598-58b9-4032-8e23-27daf0444149\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"
Oct 30 21:40:55.895714 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:55.893592    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c5a0598-58b9-4032-8e23-27daf0444149-workload-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-hkg2v\" (UID: \"2c5a0598-58b9-4032-8e23-27daf0444149\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"
Oct 30 21:40:55.895714 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:55.893633    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c5a0598-58b9-4032-8e23-27daf0444149-istio-data\") pod \"maas-default-gateway-openshift-default-7df854bf67-hkg2v\" (UID: \"2c5a0598-58b9-4032-8e23-27daf0444149\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"
Oct 30 21:40:55.895714 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:55.894026    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c5a0598-58b9-4032-8e23-27daf0444149-istio-data\") pod \"maas-default-gateway-openshift-default-7df854bf67-hkg2v\" (UID: \"2c5a0598-58b9-4032-8e23-27daf0444149\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"
Oct 30 21:40:55.895714 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:55.894544    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c5a0598-58b9-4032-8e23-27daf0444149-credential-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-hkg2v\" (UID: \"2c5a0598-58b9-4032-8e23-27daf0444149\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"
Oct 30 21:40:55.895714 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:55.895322    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c5a0598-58b9-4032-8e23-27daf0444149-workload-certs\") pod \"maas-default-gateway-openshift-default-7df854bf67-hkg2v\" (UID: \"2c5a0598-58b9-4032-8e23-27daf0444149\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"
Oct 30 21:40:55.895714 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:55.895670    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c5a0598-58b9-4032-8e23-27daf0444149-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7df854bf67-hkg2v\" (UID: \"2c5a0598-58b9-4032-8e23-27daf0444149\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"
Oct 30 21:40:55.895714 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:55.895694    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c5a0598-58b9-4032-8e23-27daf0444149-workload-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-hkg2v\" (UID: \"2c5a0598-58b9-4032-8e23-27daf0444149\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"
Oct 30 21:40:55.903278 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:55.903200    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c5a0598-58b9-4032-8e23-27daf0444149-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7df854bf67-hkg2v\" (UID: \"2c5a0598-58b9-4032-8e23-27daf0444149\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"
Oct 30 21:40:55.905076 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:55.905050    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c5a0598-58b9-4032-8e23-27daf0444149-istio-envoy\") pod \"maas-default-gateway-openshift-default-7df854bf67-hkg2v\" (UID: \"2c5a0598-58b9-4032-8e23-27daf0444149\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"
Oct 30 21:40:55.960768 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:55.960655    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nn9gc\" (UniqueName: \"kubernetes.io/projected/2c5a0598-58b9-4032-8e23-27daf0444149-kube-api-access-nn9gc\") pod \"maas-default-gateway-openshift-default-7df854bf67-hkg2v\" (UID: \"2c5a0598-58b9-4032-8e23-27daf0444149\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"
Oct 30 21:40:55.962693 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:55.962630    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c5a0598-58b9-4032-8e23-27daf0444149-istio-token\") pod \"maas-default-gateway-openshift-default-7df854bf67-hkg2v\" (UID: \"2c5a0598-58b9-4032-8e23-27daf0444149\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"
Oct 30 21:40:55.984186 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:55.982446    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-668d64f647-59zws"]
Oct 30 21:40:55.987532 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:55.987505    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-668d64f647-59zws"
Oct 30 21:40:56.001042 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.001018    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"
Oct 30 21:40:56.043881 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.043741    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-668d64f647-59zws"]
Oct 30 21:40:56.044309 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.044266    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"openshift-ai-inference-openshift-ai-inference-dockercfg-2hcrr\""
Oct 30 21:40:56.073201 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.073165    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:56.095609 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.095568    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54b7cb62-6e43-40d4-afbb-2267c98b8c52-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-668d64f647-59zws\" (UID: \"54b7cb62-6e43-40d4-afbb-2267c98b8c52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-668d64f647-59zws"
Oct 30 21:40:56.095804 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.095649    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54b7cb62-6e43-40d4-afbb-2267c98b8c52-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-668d64f647-59zws\" (UID: \"54b7cb62-6e43-40d4-afbb-2267c98b8c52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-668d64f647-59zws"
Oct 30 21:40:56.095804 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.095676    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54b7cb62-6e43-40d4-afbb-2267c98b8c52-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-668d64f647-59zws\" (UID: \"54b7cb62-6e43-40d4-afbb-2267c98b8c52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-668d64f647-59zws"
Oct 30 21:40:56.095804 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.095724    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54b7cb62-6e43-40d4-afbb-2267c98b8c52-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-668d64f647-59zws\" (UID: \"54b7cb62-6e43-40d4-afbb-2267c98b8c52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-668d64f647-59zws"
Oct 30 21:40:56.095804 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.095760    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54b7cb62-6e43-40d4-afbb-2267c98b8c52-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-668d64f647-59zws\" (UID: \"54b7cb62-6e43-40d4-afbb-2267c98b8c52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-668d64f647-59zws"
Oct 30 21:40:56.095804 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.095780    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54b7cb62-6e43-40d4-afbb-2267c98b8c52-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-668d64f647-59zws\" (UID: \"54b7cb62-6e43-40d4-afbb-2267c98b8c52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-668d64f647-59zws"
Oct 30 21:40:56.095804 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.095801    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54b7cb62-6e43-40d4-afbb-2267c98b8c52-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-668d64f647-59zws\" (UID: \"54b7cb62-6e43-40d4-afbb-2267c98b8c52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-668d64f647-59zws"
Oct 30 21:40:56.096103 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.095834    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54b7cb62-6e43-40d4-afbb-2267c98b8c52-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-668d64f647-59zws\" (UID: \"54b7cb62-6e43-40d4-afbb-2267c98b8c52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-668d64f647-59zws"
Oct 30 21:40:56.096103 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.095869    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrcvk\" (UniqueName: \"kubernetes.io/projected/54b7cb62-6e43-40d4-afbb-2267c98b8c52-kube-api-access-hrcvk\") pod \"openshift-ai-inference-openshift-ai-inference-668d64f647-59zws\" (UID: \"54b7cb62-6e43-40d4-afbb-2267c98b8c52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-668d64f647-59zws"
Oct 30 21:40:56.196963 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.196924    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54b7cb62-6e43-40d4-afbb-2267c98b8c52-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-668d64f647-59zws\" (UID: \"54b7cb62-6e43-40d4-afbb-2267c98b8c52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-668d64f647-59zws"
Oct 30 21:40:56.197133 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.196974    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54b7cb62-6e43-40d4-afbb-2267c98b8c52-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-668d64f647-59zws\" (UID: \"54b7cb62-6e43-40d4-afbb-2267c98b8c52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-668d64f647-59zws"
Oct 30 21:40:56.197133 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.197019    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54b7cb62-6e43-40d4-afbb-2267c98b8c52-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-668d64f647-59zws\" (UID: \"54b7cb62-6e43-40d4-afbb-2267c98b8c52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-668d64f647-59zws"
Oct 30 21:40:56.197133 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.197053    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54b7cb62-6e43-40d4-afbb-2267c98b8c52-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-668d64f647-59zws\" (UID: \"54b7cb62-6e43-40d4-afbb-2267c98b8c52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-668d64f647-59zws"
Oct 30 21:40:56.197133 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.197076    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54b7cb62-6e43-40d4-afbb-2267c98b8c52-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-668d64f647-59zws\" (UID: \"54b7cb62-6e43-40d4-afbb-2267c98b8c52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-668d64f647-59zws"
Oct 30 21:40:56.197133 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.197099    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54b7cb62-6e43-40d4-afbb-2267c98b8c52-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-668d64f647-59zws\" (UID: \"54b7cb62-6e43-40d4-afbb-2267c98b8c52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-668d64f647-59zws"
Oct 30 21:40:56.197444 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.197133    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54b7cb62-6e43-40d4-afbb-2267c98b8c52-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-668d64f647-59zws\" (UID: \"54b7cb62-6e43-40d4-afbb-2267c98b8c52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-668d64f647-59zws"
Oct 30 21:40:56.197444 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.197161    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrcvk\" (UniqueName: \"kubernetes.io/projected/54b7cb62-6e43-40d4-afbb-2267c98b8c52-kube-api-access-hrcvk\") pod \"openshift-ai-inference-openshift-ai-inference-668d64f647-59zws\" (UID: \"54b7cb62-6e43-40d4-afbb-2267c98b8c52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-668d64f647-59zws"
Oct 30 21:40:56.197444 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.197210    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54b7cb62-6e43-40d4-afbb-2267c98b8c52-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-668d64f647-59zws\" (UID: \"54b7cb62-6e43-40d4-afbb-2267c98b8c52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-668d64f647-59zws"
Oct 30 21:40:56.197787 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.197766    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54b7cb62-6e43-40d4-afbb-2267c98b8c52-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-668d64f647-59zws\" (UID: \"54b7cb62-6e43-40d4-afbb-2267c98b8c52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-668d64f647-59zws"
Oct 30 21:40:56.198986 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.198960    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54b7cb62-6e43-40d4-afbb-2267c98b8c52-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-668d64f647-59zws\" (UID: \"54b7cb62-6e43-40d4-afbb-2267c98b8c52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-668d64f647-59zws"
Oct 30 21:40:56.199213 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.199190    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54b7cb62-6e43-40d4-afbb-2267c98b8c52-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-668d64f647-59zws\" (UID: \"54b7cb62-6e43-40d4-afbb-2267c98b8c52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-668d64f647-59zws"
Oct 30 21:40:56.199926 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.199901    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54b7cb62-6e43-40d4-afbb-2267c98b8c52-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-668d64f647-59zws\" (UID: \"54b7cb62-6e43-40d4-afbb-2267c98b8c52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-668d64f647-59zws"
Oct 30 21:40:56.200868 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.200846    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54b7cb62-6e43-40d4-afbb-2267c98b8c52-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-668d64f647-59zws\" (UID: \"54b7cb62-6e43-40d4-afbb-2267c98b8c52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-668d64f647-59zws"
Oct 30 21:40:56.203530 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.203506    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54b7cb62-6e43-40d4-afbb-2267c98b8c52-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-668d64f647-59zws\" (UID: \"54b7cb62-6e43-40d4-afbb-2267c98b8c52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-668d64f647-59zws"
Oct 30 21:40:56.205320 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.205278    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54b7cb62-6e43-40d4-afbb-2267c98b8c52-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-668d64f647-59zws\" (UID: \"54b7cb62-6e43-40d4-afbb-2267c98b8c52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-668d64f647-59zws"
Oct 30 21:40:56.230080 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.229947    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:56.261293 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.261197    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrcvk\" (UniqueName: \"kubernetes.io/projected/54b7cb62-6e43-40d4-afbb-2267c98b8c52-kube-api-access-hrcvk\") pod \"openshift-ai-inference-openshift-ai-inference-668d64f647-59zws\" (UID: \"54b7cb62-6e43-40d4-afbb-2267c98b8c52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-668d64f647-59zws"
Oct 30 21:40:56.267911 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.267768    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54b7cb62-6e43-40d4-afbb-2267c98b8c52-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-668d64f647-59zws\" (UID: \"54b7cb62-6e43-40d4-afbb-2267c98b8c52\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-668d64f647-59zws"
Oct 30 21:40:56.309683 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.309310    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-668d64f647-59zws"
Oct 30 21:40:56.456063 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.449972    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x5g46"]
Oct 30 21:40:56.456063 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.455504    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x5g46"
Oct 30 21:40:56.483602 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.483528    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axbv8k"]
Oct 30 21:40:56.488525 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.488484    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axbv8k"
Oct 30 21:40:56.509301 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.509271    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x5g46"]
Oct 30 21:40:56.529601 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.524649    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gcgr7"]
Oct 30 21:40:56.529601 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.529267    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gcgr7"
Oct 30 21:40:56.540258 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.540205    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"]
Oct 30 21:40:56.544986 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.544868    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axbv8k"]
Oct 30 21:40:56.549779 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:40:56.548576    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c5a0598_58b9_4032_8e23_27daf0444149.slice/crio-ad56eba65ee54bb74ff31199b56b2486516b6d72e8055137eba428bffb07b8da WatchSource:0}: Error finding container ad56eba65ee54bb74ff31199b56b2486516b6d72e8055137eba428bffb07b8da: Status 404 returned error can't find the container with id ad56eba65ee54bb74ff31199b56b2486516b6d72e8055137eba428bffb07b8da
Oct 30 21:40:56.579415 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.576907    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cghp2w"]
Oct 30 21:40:56.584697 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.584132    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gcgr7"]
Oct 30 21:40:56.584697 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.584276    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cghp2w"
Oct 30 21:40:56.602163 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.601681    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/30065d64-eb7a-485a-97d6-9997da5c70fd-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x5g46\" (UID: \"30065d64-eb7a-485a-97d6-9997da5c70fd\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x5g46"
Oct 30 21:40:56.602163 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.601749    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scdsx\" (UniqueName: \"kubernetes.io/projected/9103c07d-5a90-4736-a6a6-d6f874fc1152-kube-api-access-scdsx\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axbv8k\" (UID: \"9103c07d-5a90-4736-a6a6-d6f874fc1152\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axbv8k"
Oct 30 21:40:56.602163 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.601814    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/30065d64-eb7a-485a-97d6-9997da5c70fd-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x5g46\" (UID: \"30065d64-eb7a-485a-97d6-9997da5c70fd\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x5g46"
Oct 30 21:40:56.602163 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.601843    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9103c07d-5a90-4736-a6a6-d6f874fc1152-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axbv8k\" (UID: \"9103c07d-5a90-4736-a6a6-d6f874fc1152\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axbv8k"
Oct 30 21:40:56.602163 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.601868    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9103c07d-5a90-4736-a6a6-d6f874fc1152-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axbv8k\" (UID: \"9103c07d-5a90-4736-a6a6-d6f874fc1152\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axbv8k"
Oct 30 21:40:56.602163 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.601916    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sb7hv\" (UniqueName: \"kubernetes.io/projected/30065d64-eb7a-485a-97d6-9997da5c70fd-kube-api-access-sb7hv\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x5g46\" (UID: \"30065d64-eb7a-485a-97d6-9997da5c70fd\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x5g46"
Oct 30 21:40:56.638273 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.636294    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cghp2w"]
Oct 30 21:40:56.702838 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.702794    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5qvf\" (UniqueName: \"kubernetes.io/projected/d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2-kube-api-access-b5qvf\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cghp2w\" (UID: \"d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cghp2w"
Oct 30 21:40:56.703016 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.702854    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cghp2w\" (UID: \"d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cghp2w"
Oct 30 21:40:56.703016 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.702898    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scdsx\" (UniqueName: \"kubernetes.io/projected/9103c07d-5a90-4736-a6a6-d6f874fc1152-kube-api-access-scdsx\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axbv8k\" (UID: \"9103c07d-5a90-4736-a6a6-d6f874fc1152\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axbv8k"
Oct 30 21:40:56.703016 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.702939    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c82c2960-1446-4516-bbe9-dc7886be330d-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gcgr7\" (UID: \"c82c2960-1446-4516-bbe9-dc7886be330d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gcgr7"
Oct 30 21:40:56.703131 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.703017    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/30065d64-eb7a-485a-97d6-9997da5c70fd-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x5g46\" (UID: \"30065d64-eb7a-485a-97d6-9997da5c70fd\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x5g46"
Oct 30 21:40:56.703131 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.703044    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cghp2w\" (UID: \"d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cghp2w"
Oct 30 21:40:56.703131 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.703077    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9103c07d-5a90-4736-a6a6-d6f874fc1152-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axbv8k\" (UID: \"9103c07d-5a90-4736-a6a6-d6f874fc1152\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axbv8k"
Oct 30 21:40:56.703131 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.703103    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9103c07d-5a90-4736-a6a6-d6f874fc1152-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axbv8k\" (UID: \"9103c07d-5a90-4736-a6a6-d6f874fc1152\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axbv8k"
Oct 30 21:40:56.703348 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.703138    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8qr2\" (UniqueName: \"kubernetes.io/projected/c82c2960-1446-4516-bbe9-dc7886be330d-kube-api-access-d8qr2\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gcgr7\" (UID: \"c82c2960-1446-4516-bbe9-dc7886be330d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gcgr7"
Oct 30 21:40:56.703348 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.703185    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sb7hv\" (UniqueName: \"kubernetes.io/projected/30065d64-eb7a-485a-97d6-9997da5c70fd-kube-api-access-sb7hv\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x5g46\" (UID: \"30065d64-eb7a-485a-97d6-9997da5c70fd\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x5g46"
Oct 30 21:40:56.703348 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.703318    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c82c2960-1446-4516-bbe9-dc7886be330d-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gcgr7\" (UID: \"c82c2960-1446-4516-bbe9-dc7886be330d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gcgr7"
Oct 30 21:40:56.703503 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.703354    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/30065d64-eb7a-485a-97d6-9997da5c70fd-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x5g46\" (UID: \"30065d64-eb7a-485a-97d6-9997da5c70fd\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x5g46"
Oct 30 21:40:56.705647 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.703876    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/30065d64-eb7a-485a-97d6-9997da5c70fd-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x5g46\" (UID: \"30065d64-eb7a-485a-97d6-9997da5c70fd\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x5g46"
Oct 30 21:40:56.705647 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.704629    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/30065d64-eb7a-485a-97d6-9997da5c70fd-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x5g46\" (UID: \"30065d64-eb7a-485a-97d6-9997da5c70fd\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x5g46"
Oct 30 21:40:56.705647 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.704971    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9103c07d-5a90-4736-a6a6-d6f874fc1152-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axbv8k\" (UID: \"9103c07d-5a90-4736-a6a6-d6f874fc1152\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axbv8k"
Oct 30 21:40:56.705647 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.705326    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9103c07d-5a90-4736-a6a6-d6f874fc1152-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axbv8k\" (UID: \"9103c07d-5a90-4736-a6a6-d6f874fc1152\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axbv8k"
Oct 30 21:40:56.726538 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.726489    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-668d64f647-59zws"]
Oct 30 21:40:56.733216 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:40:56.733182    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54b7cb62_6e43_40d4_afbb_2267c98b8c52.slice/crio-32c061acee7a960813bd31326d57a4c2b6d9d3c1c4e3b0ebd97551d0afc93012 WatchSource:0}: Error finding container 32c061acee7a960813bd31326d57a4c2b6d9d3c1c4e3b0ebd97551d0afc93012: Status 404 returned error can't find the container with id 32c061acee7a960813bd31326d57a4c2b6d9d3c1c4e3b0ebd97551d0afc93012
Oct 30 21:40:56.769036 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.768983    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sb7hv\" (UniqueName: \"kubernetes.io/projected/30065d64-eb7a-485a-97d6-9997da5c70fd-kube-api-access-sb7hv\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x5g46\" (UID: \"30065d64-eb7a-485a-97d6-9997da5c70fd\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x5g46"
Oct 30 21:40:56.771514 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.771457    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-scdsx\" (UniqueName: \"kubernetes.io/projected/9103c07d-5a90-4736-a6a6-d6f874fc1152-kube-api-access-scdsx\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axbv8k\" (UID: \"9103c07d-5a90-4736-a6a6-d6f874fc1152\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axbv8k"
Oct 30 21:40:56.779111 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.779085    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x5g46"
Oct 30 21:40:56.804698 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.804582    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c82c2960-1446-4516-bbe9-dc7886be330d-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gcgr7\" (UID: \"c82c2960-1446-4516-bbe9-dc7886be330d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gcgr7"
Oct 30 21:40:56.804698 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.804635    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cghp2w\" (UID: \"d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cghp2w"
Oct 30 21:40:56.804698 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.804674    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8qr2\" (UniqueName: \"kubernetes.io/projected/c82c2960-1446-4516-bbe9-dc7886be330d-kube-api-access-d8qr2\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gcgr7\" (UID: \"c82c2960-1446-4516-bbe9-dc7886be330d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gcgr7"
Oct 30 21:40:56.805291 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.805141    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c82c2960-1446-4516-bbe9-dc7886be330d-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gcgr7\" (UID: \"c82c2960-1446-4516-bbe9-dc7886be330d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gcgr7"
Oct 30 21:40:56.805472 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.805434    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c82c2960-1446-4516-bbe9-dc7886be330d-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gcgr7\" (UID: \"c82c2960-1446-4516-bbe9-dc7886be330d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gcgr7"
Oct 30 21:40:56.805560 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.805525    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5qvf\" (UniqueName: \"kubernetes.io/projected/d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2-kube-api-access-b5qvf\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cghp2w\" (UID: \"d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cghp2w"
Oct 30 21:40:56.805619 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.805558    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cghp2w\" (UID: \"d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cghp2w"
Oct 30 21:40:56.806332 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.805911    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cghp2w\" (UID: \"d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cghp2w"
Oct 30 21:40:56.806332 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.806273    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cghp2w\" (UID: \"d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cghp2w"
Oct 30 21:40:56.806947 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.806925    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c82c2960-1446-4516-bbe9-dc7886be330d-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gcgr7\" (UID: \"c82c2960-1446-4516-bbe9-dc7886be330d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gcgr7"
Oct 30 21:40:56.817578 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.817553    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axbv8k"
Oct 30 21:40:56.874013 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.873944    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8qr2\" (UniqueName: \"kubernetes.io/projected/c82c2960-1446-4516-bbe9-dc7886be330d-kube-api-access-d8qr2\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gcgr7\" (UID: \"c82c2960-1446-4516-bbe9-dc7886be330d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gcgr7"
Oct 30 21:40:56.878978 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.878867    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v" event={"ID":"2c5a0598-58b9-4032-8e23-27daf0444149","Type":"ContainerStarted","Data":"ad56eba65ee54bb74ff31199b56b2486516b6d72e8055137eba428bffb07b8da"}
Oct 30 21:40:56.896093 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.894647    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5qvf\" (UniqueName: \"kubernetes.io/projected/d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2-kube-api-access-b5qvf\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cghp2w\" (UID: \"d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cghp2w"
Oct 30 21:40:56.908309 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.902463    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-668d64f647-59zws" event={"ID":"54b7cb62-6e43-40d4-afbb-2267c98b8c52","Type":"ContainerStarted","Data":"32c061acee7a960813bd31326d57a4c2b6d9d3c1c4e3b0ebd97551d0afc93012"}
Oct 30 21:40:56.912107 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:56.911915    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cghp2w"
Oct 30 21:40:57.155375 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:57.155205    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gcgr7"
Oct 30 21:40:57.202215 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:57.202098    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x5g46"]
Oct 30 21:40:57.212169 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:40:57.212135    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30065d64_eb7a_485a_97d6_9997da5c70fd.slice/crio-ce32efbc0cda2caaedddd1f11062b4fd3b22c7faecb01966e592bf24d97ac445 WatchSource:0}: Error finding container ce32efbc0cda2caaedddd1f11062b4fd3b22c7faecb01966e592bf24d97ac445: Status 404 returned error can't find the container with id ce32efbc0cda2caaedddd1f11062b4fd3b22c7faecb01966e592bf24d97ac445
Oct 30 21:40:57.278417 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:57.276536    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axbv8k"]
Oct 30 21:40:57.281575 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:40:57.280934    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9103c07d_5a90_4736_a6a6_d6f874fc1152.slice/crio-306708b6cda8f6ebf61916b8a811b3dac8d9b08a1e4bb995b01f32c82205f7c2 WatchSource:0}: Error finding container 306708b6cda8f6ebf61916b8a811b3dac8d9b08a1e4bb995b01f32c82205f7c2: Status 404 returned error can't find the container with id 306708b6cda8f6ebf61916b8a811b3dac8d9b08a1e4bb995b01f32c82205f7c2
Oct 30 21:40:57.313796 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:57.311958    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cghp2w"]
Oct 30 21:40:57.324585 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:40:57.324527    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd52b8b18_2a5c_43b0_8473_aa55cf6bd4c2.slice/crio-de075a6196aac0aa85546890f3153e4d986e1011b5c94381b5af6ac7b7174832 WatchSource:0}: Error finding container de075a6196aac0aa85546890f3153e4d986e1011b5c94381b5af6ac7b7174832: Status 404 returned error can't find the container with id de075a6196aac0aa85546890f3153e4d986e1011b5c94381b5af6ac7b7174832
Oct 30 21:40:57.633727 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:57.633642    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gcgr7"]
Oct 30 21:40:57.639940 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:40:57.639907    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc82c2960_1446_4516_bbe9_dc7886be330d.slice/crio-8aeca314aac99557d7a470b1aa0c7a5e0ed121009176e79159f03db5cddcd9ea WatchSource:0}: Error finding container 8aeca314aac99557d7a470b1aa0c7a5e0ed121009176e79159f03db5cddcd9ea: Status 404 returned error can't find the container with id 8aeca314aac99557d7a470b1aa0c7a5e0ed121009176e79159f03db5cddcd9ea
Oct 30 21:40:57.923216 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:57.923127    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cghp2w" event={"ID":"d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2","Type":"ContainerStarted","Data":"de075a6196aac0aa85546890f3153e4d986e1011b5c94381b5af6ac7b7174832"}
Oct 30 21:40:57.926158 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:57.926128    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axbv8k" event={"ID":"9103c07d-5a90-4736-a6a6-d6f874fc1152","Type":"ContainerStarted","Data":"306708b6cda8f6ebf61916b8a811b3dac8d9b08a1e4bb995b01f32c82205f7c2"}
Oct 30 21:40:57.933272 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:57.933215    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x5g46" event={"ID":"30065d64-eb7a-485a-97d6-9997da5c70fd","Type":"ContainerStarted","Data":"ce32efbc0cda2caaedddd1f11062b4fd3b22c7faecb01966e592bf24d97ac445"}
Oct 30 21:40:57.939588 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:40:57.939477    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gcgr7" event={"ID":"c82c2960-1446-4516-bbe9-dc7886be330d","Type":"ContainerStarted","Data":"8aeca314aac99557d7a470b1aa0c7a5e0ed121009176e79159f03db5cddcd9ea"}
Oct 30 21:41:00.057638 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:00.057537    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:02.663432 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:02.663399    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:05.663254 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:41:05.663151    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:41:09.086935 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:09.086888    4871 generic.go:356] "Generic (PLEG): container finished" podID="56a9d68e-fdea-4bd4-a96c-855199901791" containerID="83eb519e830b50b52849882f407e57be5536f8e606141f11695fdb58d9975137" exitCode=0
Oct 30 21:41:09.089115 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:09.089084    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ndg4g" event={"ID":"56a9d68e-fdea-4bd4-a96c-855199901791","Type":"ContainerDied","Data":"83eb519e830b50b52849882f407e57be5536f8e606141f11695fdb58d9975137"}
Oct 30 21:41:09.686665 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:09.686617    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:41:09.686858 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:09.686733    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:41:09.686858 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:09.686777    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:41:10.099184 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:10.097573    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx" event={"ID":"8d401654-8a4c-4a79-b2d3-612765312421","Type":"ContainerStarted","Data":"177454af51ef86d77b2e5de2f249dbdade385ac7dc2d4156d48f5d3999c4b7c8"}
Oct 30 21:41:10.105205 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:10.105165    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-668d64f647-59zws" event={"ID":"54b7cb62-6e43-40d4-afbb-2267c98b8c52","Type":"ContainerStarted","Data":"02077c3db22b992af5d33a92b5ce40280a2e149861266a6c953341e8f3238040"}
Oct 30 21:41:10.310325 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:10.310207    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-668d64f647-59zws"
Oct 30 21:41:10.317172 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:10.317135    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-668d64f647-59zws"
Oct 30 21:41:10.330736 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:10.329335    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-668d64f647-59zws" podStartSLOduration=2.391942128 podStartE2EDuration="15.32931454s" totalImagesPullingTime="12.937372412s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:40:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:41:10.328089889 +0000 UTC m=+212.505578507" watchObservedRunningTime="2026-10-30 21:41:10.32931454 +0000 UTC m=+212.506803155"
Oct 30 21:41:10.798610 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:10.798559    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:41:10.798815 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:10.798656    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:41:10.798815 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:10.798702    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:41:11.117616 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:11.117578    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v" event={"ID":"2c5a0598-58b9-4032-8e23-27daf0444149","Type":"ContainerStarted","Data":"4c2b14109615eae580160381e381270bccd8598b240e30f91c9ee2f4c7961fa3"}
Oct 30 21:41:11.122310 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:11.122280    4871 generic.go:356] "Generic (PLEG): container finished" podID="8d401654-8a4c-4a79-b2d3-612765312421" containerID="177454af51ef86d77b2e5de2f249dbdade385ac7dc2d4156d48f5d3999c4b7c8" exitCode=0
Oct 30 21:41:11.124436 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:11.124409    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx" event={"ID":"8d401654-8a4c-4a79-b2d3-612765312421","Type":"ContainerDied","Data":"177454af51ef86d77b2e5de2f249dbdade385ac7dc2d4156d48f5d3999c4b7c8"}
Oct 30 21:41:11.124608 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:11.124465    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-668d64f647-59zws"
Oct 30 21:41:11.125657 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:11.125597    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-668d64f647-59zws"
Oct 30 21:41:11.245567 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:11.244772    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v" podStartSLOduration=2.00247454 podStartE2EDuration="16.244749734s" totalImagesPullingTime="14.242275194s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:40:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:41:11.243755569 +0000 UTC m=+213.421244191" watchObservedRunningTime="2026-10-30 21:41:11.244749734 +0000 UTC m=+213.422238350"
Oct 30 21:41:12.002001 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:12.001968    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"
Oct 30 21:41:12.008385 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:12.008354    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"
Oct 30 21:41:12.136395 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:12.136355    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ndg4g" event={"ID":"56a9d68e-fdea-4bd4-a96c-855199901791","Type":"ContainerDied","Data":"76d15dc3e1f8764406f3844115443767cda9c4ccb60f69c19846484b2599683a"}
Oct 30 21:41:12.137131 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:12.137033    4871 generic.go:356] "Generic (PLEG): container finished" podID="56a9d68e-fdea-4bd4-a96c-855199901791" containerID="76d15dc3e1f8764406f3844115443767cda9c4ccb60f69c19846484b2599683a" exitCode=0
Oct 30 21:41:12.138121 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:12.138100    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"
Oct 30 21:41:12.138984 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:12.138937    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"
Oct 30 21:41:13.143693 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:13.143662    4871 generic.go:356] "Generic (PLEG): container finished" podID="30065d64-eb7a-485a-97d6-9997da5c70fd" containerID="e2446188f0f35d4497e659193a23e29c3e270a1cb69f1bc5690c188e4dd99885" exitCode=0
Oct 30 21:41:13.144041 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:13.143742    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x5g46" event={"ID":"30065d64-eb7a-485a-97d6-9997da5c70fd","Type":"ContainerDied","Data":"e2446188f0f35d4497e659193a23e29c3e270a1cb69f1bc5690c188e4dd99885"}
Oct 30 21:41:13.145430 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:13.145408    4871 generic.go:356] "Generic (PLEG): container finished" podID="cfdfd8e2-c269-41a5-847f-2760cd6577c7" containerID="0a3dc5c96b924b6cdb6e6a153507a65dfc764629abcc83605c62e34ed2d54a30" exitCode=0
Oct 30 21:41:13.145623 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:13.145549    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agzz78" event={"ID":"cfdfd8e2-c269-41a5-847f-2760cd6577c7","Type":"ContainerDied","Data":"0a3dc5c96b924b6cdb6e6a153507a65dfc764629abcc83605c62e34ed2d54a30"}
Oct 30 21:41:13.148046 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:13.147646    4871 generic.go:356] "Generic (PLEG): container finished" podID="8d401654-8a4c-4a79-b2d3-612765312421" containerID="eb28e7de2c177ff654e1ba8b483c22cc2f4e0f01c81bc8d545481db5ceddd2f3" exitCode=0
Oct 30 21:41:13.148046 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:13.147736    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx" event={"ID":"8d401654-8a4c-4a79-b2d3-612765312421","Type":"ContainerDied","Data":"eb28e7de2c177ff654e1ba8b483c22cc2f4e0f01c81bc8d545481db5ceddd2f3"}
Oct 30 21:41:14.153133 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:14.153100    4871 generic.go:356] "Generic (PLEG): container finished" podID="53353129-6959-43a8-83c5-e845c7869516" containerID="0920f4929bda8aa6c7bdacd3d0d2953983f38d24645cbb535b63f92edd81096b" exitCode=0
Oct 30 21:41:14.153632 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:14.153178    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vsdmv" event={"ID":"53353129-6959-43a8-83c5-e845c7869516","Type":"ContainerDied","Data":"0920f4929bda8aa6c7bdacd3d0d2953983f38d24645cbb535b63f92edd81096b"}
Oct 30 21:41:14.154776 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:14.154754    4871 generic.go:356] "Generic (PLEG): container finished" podID="c82c2960-1446-4516-bbe9-dc7886be330d" containerID="59996867834bf46a60fa6cb5d812ede9c0a8f6132d8205e896291721117d7ef2" exitCode=0
Oct 30 21:41:14.154872 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:14.154839    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gcgr7" event={"ID":"c82c2960-1446-4516-bbe9-dc7886be330d","Type":"ContainerDied","Data":"59996867834bf46a60fa6cb5d812ede9c0a8f6132d8205e896291721117d7ef2"}
Oct 30 21:41:14.156135 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:14.156115    4871 generic.go:356] "Generic (PLEG): container finished" podID="51e4633e-d58b-4843-92e1-35c0bfb629d3" containerID="0d90c2b894f1455054fbad25c8e9131119f9026751e17ce1d5f22e1c35aecf29" exitCode=0
Oct 30 21:41:14.156217 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:14.156194    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6b9qt6" event={"ID":"51e4633e-d58b-4843-92e1-35c0bfb629d3","Type":"ContainerDied","Data":"0d90c2b894f1455054fbad25c8e9131119f9026751e17ce1d5f22e1c35aecf29"}
Oct 30 21:41:14.157629 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:14.157604    4871 generic.go:356] "Generic (PLEG): container finished" podID="570a9217-1578-43e4-a5fd-65c8b261b535" containerID="3c188e110a3d29b4d183792ce6ed106b09b1b7bfd5ac9e8ad6bd7b97dc209da1" exitCode=0
Oct 30 21:41:14.157687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:14.157667    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqcvcw" event={"ID":"570a9217-1578-43e4-a5fd-65c8b261b535","Type":"ContainerDied","Data":"3c188e110a3d29b4d183792ce6ed106b09b1b7bfd5ac9e8ad6bd7b97dc209da1"}
Oct 30 21:41:14.159074 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:14.159052    4871 generic.go:356] "Generic (PLEG): container finished" podID="d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2" containerID="3172e3c6d059d3e91705716e995181a5c274f0e1ac156e11b791bf53ed0e2df3" exitCode=0
Oct 30 21:41:14.159153 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:14.159123    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cghp2w" event={"ID":"d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2","Type":"ContainerDied","Data":"3172e3c6d059d3e91705716e995181a5c274f0e1ac156e11b791bf53ed0e2df3"}
Oct 30 21:41:15.166990 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:15.166955    4871 generic.go:356] "Generic (PLEG): container finished" podID="036a0620-585b-4e3f-bb7e-5f4bb7799498" containerID="823203edafb77db4a3d00b64e07cdf61cdc9878f4028ddc37047ac41955eeb39" exitCode=0
Oct 30 21:41:15.167528 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:15.167045    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fjfzw" event={"ID":"036a0620-585b-4e3f-bb7e-5f4bb7799498","Type":"ContainerDied","Data":"823203edafb77db4a3d00b64e07cdf61cdc9878f4028ddc37047ac41955eeb39"}
Oct 30 21:41:15.169794 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:15.169767    4871 generic.go:356] "Generic (PLEG): container finished" podID="689c9fec-d928-483c-aea2-d5cd577213ff" containerID="237bfec0571f1512a8fef68272e4cbd1f54b031479712c195f5a74eab6018500" exitCode=0
Oct 30 21:41:15.169908 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:15.169884    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqplt6" event={"ID":"689c9fec-d928-483c-aea2-d5cd577213ff","Type":"ContainerDied","Data":"237bfec0571f1512a8fef68272e4cbd1f54b031479712c195f5a74eab6018500"}
Oct 30 21:41:15.172348 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:15.172322    4871 generic.go:356] "Generic (PLEG): container finished" podID="9103c07d-5a90-4736-a6a6-d6f874fc1152" containerID="11fe3690b87e4c0cb48048ea46d65b1c1b569577730f8ea965ad8f37c0fcda27" exitCode=0
Oct 30 21:41:15.172457 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:15.172404    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axbv8k" event={"ID":"9103c07d-5a90-4736-a6a6-d6f874fc1152","Type":"ContainerDied","Data":"11fe3690b87e4c0cb48048ea46d65b1c1b569577730f8ea965ad8f37c0fcda27"}
Oct 30 21:41:16.183530 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:16.183494    4871 generic.go:356] "Generic (PLEG): container finished" podID="30065d64-eb7a-485a-97d6-9997da5c70fd" containerID="1db62a25a59fb724d3e2ad84baac4be88cb15f091d3493f4b94dee36fb821278" exitCode=0
Oct 30 21:41:16.183947 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:16.183612    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x5g46" event={"ID":"30065d64-eb7a-485a-97d6-9997da5c70fd","Type":"ContainerDied","Data":"1db62a25a59fb724d3e2ad84baac4be88cb15f091d3493f4b94dee36fb821278"}
Oct 30 21:41:16.187074 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:16.186877    4871 generic.go:356] "Generic (PLEG): container finished" podID="c82c2960-1446-4516-bbe9-dc7886be330d" containerID="9920c1818b0a4486d6ca28fca7cc3290b2b57667b1e387fb3a354ca957455137" exitCode=0
Oct 30 21:41:16.187074 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:16.186961    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gcgr7" event={"ID":"c82c2960-1446-4516-bbe9-dc7886be330d","Type":"ContainerDied","Data":"9920c1818b0a4486d6ca28fca7cc3290b2b57667b1e387fb3a354ca957455137"}
Oct 30 21:41:16.192931 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:16.192900    4871 generic.go:356] "Generic (PLEG): container finished" podID="51e4633e-d58b-4843-92e1-35c0bfb629d3" containerID="ca44ce66e8e8d58b9d415d19638834c6078eee988bd6ad0e7a909bf8adb16d41" exitCode=0
Oct 30 21:41:16.193068 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:16.193049    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6b9qt6" event={"ID":"51e4633e-d58b-4843-92e1-35c0bfb629d3","Type":"ContainerDied","Data":"ca44ce66e8e8d58b9d415d19638834c6078eee988bd6ad0e7a909bf8adb16d41"}
Oct 30 21:41:16.197427 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:16.197399    4871 generic.go:356] "Generic (PLEG): container finished" podID="d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2" containerID="a4fed501764d9e2a596bff7a9caffc039e179b7d27d440f3dc4b272c34ebd359" exitCode=0
Oct 30 21:41:16.197544 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:16.197466    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cghp2w" event={"ID":"d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2","Type":"ContainerDied","Data":"a4fed501764d9e2a596bff7a9caffc039e179b7d27d440f3dc4b272c34ebd359"}
Oct 30 21:41:17.215852 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:17.214800    4871 generic.go:356] "Generic (PLEG): container finished" podID="689c9fec-d928-483c-aea2-d5cd577213ff" containerID="93f0a43a8d687b6cbdbf1cc96431e9e5f983f37487c5413088234b994afe77aa" exitCode=0
Oct 30 21:41:17.215852 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:17.214929    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqplt6" event={"ID":"689c9fec-d928-483c-aea2-d5cd577213ff","Type":"ContainerDied","Data":"93f0a43a8d687b6cbdbf1cc96431e9e5f983f37487c5413088234b994afe77aa"}
Oct 30 21:41:17.222492 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:17.221661    4871 generic.go:356] "Generic (PLEG): container finished" podID="9103c07d-5a90-4736-a6a6-d6f874fc1152" containerID="60ce1005dc571d8588cba7f522e10b0af3e9ffad58f98926bc91a3e67693c1ef" exitCode=0
Oct 30 21:41:17.222492 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:17.221786    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axbv8k" event={"ID":"9103c07d-5a90-4736-a6a6-d6f874fc1152","Type":"ContainerDied","Data":"60ce1005dc571d8588cba7f522e10b0af3e9ffad58f98926bc91a3e67693c1ef"}
Oct 30 21:41:17.226416 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:17.225580    4871 generic.go:356] "Generic (PLEG): container finished" podID="cfdfd8e2-c269-41a5-847f-2760cd6577c7" containerID="e65959da1a1d89d39470e154a53eef87ed27c8bb8f0795b91aa2d5362d2f22f3" exitCode=0
Oct 30 21:41:17.226416 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:17.226351    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agzz78" event={"ID":"cfdfd8e2-c269-41a5-847f-2760cd6577c7","Type":"ContainerDied","Data":"e65959da1a1d89d39470e154a53eef87ed27c8bb8f0795b91aa2d5362d2f22f3"}
Oct 30 21:41:17.229060 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:17.229029    4871 generic.go:356] "Generic (PLEG): container finished" podID="036a0620-585b-4e3f-bb7e-5f4bb7799498" containerID="99a1b94402eca6037e0241f21e6feab593a74e81934f1874c43d1b905b6295d1" exitCode=0
Oct 30 21:41:17.229162 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:17.229077    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fjfzw" event={"ID":"036a0620-585b-4e3f-bb7e-5f4bb7799498","Type":"ContainerDied","Data":"99a1b94402eca6037e0241f21e6feab593a74e81934f1874c43d1b905b6295d1"}
Oct 30 21:41:20.279065 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:20.279022    4871 generic.go:356] "Generic (PLEG): container finished" podID="53353129-6959-43a8-83c5-e845c7869516" containerID="d1b0b0d38fc080793f6ba180c2ade781117fcbeafb6c1e70b8dd3419e2b9eddd" exitCode=0
Oct 30 21:41:20.279550 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:20.279214    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vsdmv" event={"ID":"53353129-6959-43a8-83c5-e845c7869516","Type":"ContainerDied","Data":"d1b0b0d38fc080793f6ba180c2ade781117fcbeafb6c1e70b8dd3419e2b9eddd"}
Oct 30 21:41:22.932651 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:22.932609    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:23.091164 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:23.091032    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:23.352163 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:23.351949    4871 generic.go:356] "Generic (PLEG): container finished" podID="570a9217-1578-43e4-a5fd-65c8b261b535" containerID="77b53836185182f602c69cc6098a4ca611e8dd7b82a87f4b5afd33891b044f48" exitCode=0
Oct 30 21:41:23.352163 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:23.352069    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqcvcw" event={"ID":"570a9217-1578-43e4-a5fd-65c8b261b535","Type":"ContainerDied","Data":"77b53836185182f602c69cc6098a4ca611e8dd7b82a87f4b5afd33891b044f48"}
Oct 30 21:41:25.908750 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:25.908705    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:26.063048 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:26.062941    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:28.982781 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:28.982747    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:35.597657 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:41:35.597612    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:41:36.508353 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:36.507401    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx" event={"ID":"8d401654-8a4c-4a79-b2d3-612765312421","Type":"ContainerStarted","Data":"902915ab0d38ea6fcd1fdbb73c2ca3ffe7039518f0e32af1aa6688365d59619c"}
Oct 30 21:41:36.514702 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:36.514101    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ndg4g" event={"ID":"56a9d68e-fdea-4bd4-a96c-855199901791","Type":"ContainerStarted","Data":"ef5cc83d56e3a75c55439e24be73f14945af385193b9e59609f0ad84bf14aa9c"}
Oct 30 21:41:36.623088 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:36.620930    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx" podStartSLOduration=4.318728967 podStartE2EDuration="43.62090806s" totalImagesPullingTime="39.302179093s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:40:53 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:41:36.620384334 +0000 UTC m=+238.797872954" watchObservedRunningTime="2026-10-30 21:41:36.62090806 +0000 UTC m=+238.798396678"
Oct 30 21:41:36.735641 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:36.735568    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ndg4g" podStartSLOduration=3.788755132 podStartE2EDuration="48.735546993s" totalImagesPullingTime="44.946791861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:40:48 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:41:36.73214095 +0000 UTC m=+238.909629567" watchObservedRunningTime="2026-10-30 21:41:36.735546993 +0000 UTC m=+238.913035606"
Oct 30 21:41:37.542952 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:37.542888    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx_8d401654-8a4c-4a79-b2d3-612765312421/extract/0.log"
Oct 30 21:41:37.564006 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:37.563637    4871 generic.go:356] "Generic (PLEG): container finished" podID="8d401654-8a4c-4a79-b2d3-612765312421" containerID="902915ab0d38ea6fcd1fdbb73c2ca3ffe7039518f0e32af1aa6688365d59619c" exitCode=1
Oct 30 21:41:37.564006 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:37.563832    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx" event={"ID":"8d401654-8a4c-4a79-b2d3-612765312421","Type":"ContainerDied","Data":"902915ab0d38ea6fcd1fdbb73c2ca3ffe7039518f0e32af1aa6688365d59619c"}
Oct 30 21:41:37.576080 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:37.576038    4871 generic.go:356] "Generic (PLEG): container finished" podID="56a9d68e-fdea-4bd4-a96c-855199901791" containerID="ef5cc83d56e3a75c55439e24be73f14945af385193b9e59609f0ad84bf14aa9c" exitCode=0
Oct 30 21:41:37.576555 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:37.576494    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ndg4g" event={"ID":"56a9d68e-fdea-4bd4-a96c-855199901791","Type":"ContainerDied","Data":"ef5cc83d56e3a75c55439e24be73f14945af385193b9e59609f0ad84bf14aa9c"}
Oct 30 21:41:38.598108 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:38.595760    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqcvcw" event={"ID":"570a9217-1578-43e4-a5fd-65c8b261b535","Type":"ContainerStarted","Data":"186bb556da3e7b7409526a136de116f608defd4003cf3a73a34386258c11be1a"}
Oct 30 21:41:38.712565 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:38.711675    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqcvcw" podStartSLOduration=2.616071165 podStartE2EDuration="49.711657155s" totalImagesPullingTime="47.09558599s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:40:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:41:38.710573944 +0000 UTC m=+240.888062560" watchObservedRunningTime="2026-10-30 21:41:38.711657155 +0000 UTC m=+240.889145771"
Oct 30 21:41:38.796159 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:38.796114    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx_8d401654-8a4c-4a79-b2d3-612765312421/extract/0.log"
Oct 30 21:41:38.812936 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:38.812909    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx"
Oct 30 21:41:38.980711 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:38.980367    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ndg4g"
Oct 30 21:41:38.997129 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:38.997087    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8d401654-8a4c-4a79-b2d3-612765312421-bundle\" (UniqueName: \"kubernetes.io/empty-dir/8d401654-8a4c-4a79-b2d3-612765312421-bundle\") pod \"8d401654-8a4c-4a79-b2d3-612765312421\" (UID: \"8d401654-8a4c-4a79-b2d3-612765312421\") "
Oct 30 21:41:38.997375 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:38.997248    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8d401654-8a4c-4a79-b2d3-612765312421-kube-api-access-fn4k7\" (UniqueName: \"kubernetes.io/projected/8d401654-8a4c-4a79-b2d3-612765312421-kube-api-access-fn4k7\") pod \"8d401654-8a4c-4a79-b2d3-612765312421\" (UID: \"8d401654-8a4c-4a79-b2d3-612765312421\") "
Oct 30 21:41:38.997375 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:38.997322    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8d401654-8a4c-4a79-b2d3-612765312421-util\" (UniqueName: \"kubernetes.io/empty-dir/8d401654-8a4c-4a79-b2d3-612765312421-util\") pod \"8d401654-8a4c-4a79-b2d3-612765312421\" (UID: \"8d401654-8a4c-4a79-b2d3-612765312421\") "
Oct 30 21:41:38.998737 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:38.998688    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d401654-8a4c-4a79-b2d3-612765312421-bundle" pod "8d401654-8a4c-4a79-b2d3-612765312421" (UID: "8d401654-8a4c-4a79-b2d3-612765312421"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:39.004264 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:39.001600    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d401654-8a4c-4a79-b2d3-612765312421-kube-api-access-fn4k7" pod "8d401654-8a4c-4a79-b2d3-612765312421" (UID: "8d401654-8a4c-4a79-b2d3-612765312421"). InnerVolumeSpecName "kube-api-access-fn4k7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:41:39.005760 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:39.005724    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d401654-8a4c-4a79-b2d3-612765312421-util" pod "8d401654-8a4c-4a79-b2d3-612765312421" (UID: "8d401654-8a4c-4a79-b2d3-612765312421"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:39.100979 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:39.100318    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/56a9d68e-fdea-4bd4-a96c-855199901791-kube-api-access-kfxlq\" (UniqueName: \"kubernetes.io/projected/56a9d68e-fdea-4bd4-a96c-855199901791-kube-api-access-kfxlq\") pod \"56a9d68e-fdea-4bd4-a96c-855199901791\" (UID: \"56a9d68e-fdea-4bd4-a96c-855199901791\") "
Oct 30 21:41:39.100979 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:39.100384    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/56a9d68e-fdea-4bd4-a96c-855199901791-util\" (UniqueName: \"kubernetes.io/empty-dir/56a9d68e-fdea-4bd4-a96c-855199901791-util\") pod \"56a9d68e-fdea-4bd4-a96c-855199901791\" (UID: \"56a9d68e-fdea-4bd4-a96c-855199901791\") "
Oct 30 21:41:39.100979 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:39.100426    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/56a9d68e-fdea-4bd4-a96c-855199901791-bundle\" (UniqueName: \"kubernetes.io/empty-dir/56a9d68e-fdea-4bd4-a96c-855199901791-bundle\") pod \"56a9d68e-fdea-4bd4-a96c-855199901791\" (UID: \"56a9d68e-fdea-4bd4-a96c-855199901791\") "
Oct 30 21:41:39.100979 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:39.100653    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fn4k7\" (UniqueName: \"kubernetes.io/projected/8d401654-8a4c-4a79-b2d3-612765312421-kube-api-access-fn4k7\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:39.100979 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:39.100671    4871 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8d401654-8a4c-4a79-b2d3-612765312421-util\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:39.100979 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:39.100687    4871 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8d401654-8a4c-4a79-b2d3-612765312421-bundle\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:39.101771 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:39.101733    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56a9d68e-fdea-4bd4-a96c-855199901791-bundle" pod "56a9d68e-fdea-4bd4-a96c-855199901791" (UID: "56a9d68e-fdea-4bd4-a96c-855199901791"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:39.104909 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:39.104867    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56a9d68e-fdea-4bd4-a96c-855199901791-kube-api-access-kfxlq" pod "56a9d68e-fdea-4bd4-a96c-855199901791" (UID: "56a9d68e-fdea-4bd4-a96c-855199901791"). InnerVolumeSpecName "kube-api-access-kfxlq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:41:39.108289 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:39.108216    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56a9d68e-fdea-4bd4-a96c-855199901791-util" pod "56a9d68e-fdea-4bd4-a96c-855199901791" (UID: "56a9d68e-fdea-4bd4-a96c-855199901791"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:39.202101 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:39.201954    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kfxlq\" (UniqueName: \"kubernetes.io/projected/56a9d68e-fdea-4bd4-a96c-855199901791-kube-api-access-kfxlq\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:39.202101 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:39.201999    4871 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/56a9d68e-fdea-4bd4-a96c-855199901791-util\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:39.202101 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:39.202015    4871 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/56a9d68e-fdea-4bd4-a96c-855199901791-bundle\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:39.629020 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:39.628980    4871 generic.go:356] "Generic (PLEG): container finished" podID="570a9217-1578-43e4-a5fd-65c8b261b535" containerID="186bb556da3e7b7409526a136de116f608defd4003cf3a73a34386258c11be1a" exitCode=0
Oct 30 21:41:39.629553 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:39.629146    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqcvcw" event={"ID":"570a9217-1578-43e4-a5fd-65c8b261b535","Type":"ContainerDied","Data":"186bb556da3e7b7409526a136de116f608defd4003cf3a73a34386258c11be1a"}
Oct 30 21:41:39.633780 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:39.633719    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx_8d401654-8a4c-4a79-b2d3-612765312421/extract/0.log"
Oct 30 21:41:39.635599 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:39.635568    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx" event={"ID":"8d401654-8a4c-4a79-b2d3-612765312421","Type":"ContainerDied","Data":"aea3b7da9b7840e0c6788cb479ce0082c9d8374ce3e8fbc42667430da4c1800f"}
Oct 30 21:41:39.635749 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:39.635625    4871 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aea3b7da9b7840e0c6788cb479ce0082c9d8374ce3e8fbc42667430da4c1800f"
Oct 30 21:41:39.635749 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:39.635721    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx"
Oct 30 21:41:39.645190 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:39.645142    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ndg4g" event={"ID":"56a9d68e-fdea-4bd4-a96c-855199901791","Type":"ContainerDied","Data":"1f4af9a67f6cc9725acb2d1d98c6cabc38f20fdd8cadefc7fb47b106086f3bcf"}
Oct 30 21:41:39.645405 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:39.645207    4871 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1f4af9a67f6cc9725acb2d1d98c6cabc38f20fdd8cadefc7fb47b106086f3bcf"
Oct 30 21:41:39.645405 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:39.645307    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ndg4g"
Oct 30 21:41:40.666104 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:40.666064    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cghp2w" event={"ID":"d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2","Type":"ContainerStarted","Data":"7489005e764cb5b15be68b1948f69f015edc8c3b2beddb52a332820037368046"}
Oct 30 21:41:40.926259 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:40.925810    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqcvcw"
Oct 30 21:41:41.026281 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:41.025566    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/570a9217-1578-43e4-a5fd-65c8b261b535-util\" (UniqueName: \"kubernetes.io/empty-dir/570a9217-1578-43e4-a5fd-65c8b261b535-util\") pod \"570a9217-1578-43e4-a5fd-65c8b261b535\" (UID: \"570a9217-1578-43e4-a5fd-65c8b261b535\") "
Oct 30 21:41:41.026281 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:41.025676    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/570a9217-1578-43e4-a5fd-65c8b261b535-kube-api-access-kcglm\" (UniqueName: \"kubernetes.io/projected/570a9217-1578-43e4-a5fd-65c8b261b535-kube-api-access-kcglm\") pod \"570a9217-1578-43e4-a5fd-65c8b261b535\" (UID: \"570a9217-1578-43e4-a5fd-65c8b261b535\") "
Oct 30 21:41:41.026281 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:41.025727    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/570a9217-1578-43e4-a5fd-65c8b261b535-bundle\" (UniqueName: \"kubernetes.io/empty-dir/570a9217-1578-43e4-a5fd-65c8b261b535-bundle\") pod \"570a9217-1578-43e4-a5fd-65c8b261b535\" (UID: \"570a9217-1578-43e4-a5fd-65c8b261b535\") "
Oct 30 21:41:41.027747 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:41.027694    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/570a9217-1578-43e4-a5fd-65c8b261b535-bundle" pod "570a9217-1578-43e4-a5fd-65c8b261b535" (UID: "570a9217-1578-43e4-a5fd-65c8b261b535"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:41.032980 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:41.032938    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/570a9217-1578-43e4-a5fd-65c8b261b535-util" pod "570a9217-1578-43e4-a5fd-65c8b261b535" (UID: "570a9217-1578-43e4-a5fd-65c8b261b535"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:41.034688 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:41.034640    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/570a9217-1578-43e4-a5fd-65c8b261b535-kube-api-access-kcglm" pod "570a9217-1578-43e4-a5fd-65c8b261b535" (UID: "570a9217-1578-43e4-a5fd-65c8b261b535"). InnerVolumeSpecName "kube-api-access-kcglm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:41:41.041141 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:41.040767    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cghp2w" podStartSLOduration=3.29741903 podStartE2EDuration="45.040747192s" totalImagesPullingTime="41.743328162s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:40:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:41:40.787297298 +0000 UTC m=+242.964785914" watchObservedRunningTime="2026-10-30 21:41:41.040747192 +0000 UTC m=+243.218235809"
Oct 30 21:41:41.126787 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:41.126752    4871 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/570a9217-1578-43e4-a5fd-65c8b261b535-util\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:41.126787 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:41.126788    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kcglm\" (UniqueName: \"kubernetes.io/projected/570a9217-1578-43e4-a5fd-65c8b261b535-kube-api-access-kcglm\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:41.127021 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:41.126805    4871 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/570a9217-1578-43e4-a5fd-65c8b261b535-bundle\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:41.677717 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:41.677675    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x5g46" event={"ID":"30065d64-eb7a-485a-97d6-9997da5c70fd","Type":"ContainerStarted","Data":"a4827ad1ae6e869820e8e90f0559833d223f98f56359f9a8ee5d53c70375beab"}
Oct 30 21:41:41.681386 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:41.681347    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqcvcw" event={"ID":"570a9217-1578-43e4-a5fd-65c8b261b535","Type":"ContainerDied","Data":"fd1f8406ac78d06300f424b674eae0037a3ae931d8114f2c0aa9bd8753147e78"}
Oct 30 21:41:41.681556 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:41.681411    4871 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fd1f8406ac78d06300f424b674eae0037a3ae931d8114f2c0aa9bd8753147e78"
Oct 30 21:41:41.681556 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:41.681522    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqcvcw"
Oct 30 21:41:41.689273 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:41.689111    4871 generic.go:356] "Generic (PLEG): container finished" podID="d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2" containerID="7489005e764cb5b15be68b1948f69f015edc8c3b2beddb52a332820037368046" exitCode=0
Oct 30 21:41:41.689273 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:41.689227    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cghp2w" event={"ID":"d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2","Type":"ContainerDied","Data":"7489005e764cb5b15be68b1948f69f015edc8c3b2beddb52a332820037368046"}
Oct 30 21:41:41.793756 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:41.793144    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x5g46" podStartSLOduration=2.885991056 podStartE2EDuration="45.7930947s" totalImagesPullingTime="42.907103644s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:40:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:41:41.791074395 +0000 UTC m=+243.968563014" watchObservedRunningTime="2026-10-30 21:41:41.7930947 +0000 UTC m=+243.970583316"
Oct 30 21:41:42.697488 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:42.697452    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqplt6" event={"ID":"689c9fec-d928-483c-aea2-d5cd577213ff","Type":"ContainerStarted","Data":"ab201b45215d10adfb83cec94149783ea01eb538d95129f639dee944226b3d18"}
Oct 30 21:41:42.699902 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:42.699875    4871 generic.go:356] "Generic (PLEG): container finished" podID="30065d64-eb7a-485a-97d6-9997da5c70fd" containerID="a4827ad1ae6e869820e8e90f0559833d223f98f56359f9a8ee5d53c70375beab" exitCode=0
Oct 30 21:41:42.700010 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:42.699962    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x5g46" event={"ID":"30065d64-eb7a-485a-97d6-9997da5c70fd","Type":"ContainerDied","Data":"a4827ad1ae6e869820e8e90f0559833d223f98f56359f9a8ee5d53c70375beab"}
Oct 30 21:41:42.702621 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:42.702591    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agzz78" event={"ID":"cfdfd8e2-c269-41a5-847f-2760cd6577c7","Type":"ContainerStarted","Data":"c6d6dc197ec7513ab6d59ccdfa35dce2cf8c299d64f2c5d6286741a03aa2a1e0"}
Oct 30 21:41:42.705250 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:42.705208    4871 generic.go:356] "Generic (PLEG): container finished" podID="c82c2960-1446-4516-bbe9-dc7886be330d" containerID="801d293d54227b0288e607d7b87a988637aeccf1fe828834e5624ef260e30768" exitCode=0
Oct 30 21:41:42.705354 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:42.705301    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gcgr7" event={"ID":"c82c2960-1446-4516-bbe9-dc7886be330d","Type":"ContainerDied","Data":"801d293d54227b0288e607d7b87a988637aeccf1fe828834e5624ef260e30768"}
Oct 30 21:41:42.707858 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:42.707834    4871 generic.go:356] "Generic (PLEG): container finished" podID="51e4633e-d58b-4843-92e1-35c0bfb629d3" containerID="564c31a5245ef0f68cc5be4f7da121903edb2d93ed9c581b2cc46ebad2dca5bf" exitCode=0
Oct 30 21:41:42.707998 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:42.707927    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6b9qt6" event={"ID":"51e4633e-d58b-4843-92e1-35c0bfb629d3","Type":"ContainerDied","Data":"564c31a5245ef0f68cc5be4f7da121903edb2d93ed9c581b2cc46ebad2dca5bf"}
Oct 30 21:41:42.811304 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:42.811251    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqplt6" podStartSLOduration=3.239994629 podStartE2EDuration="53.811222591s" totalImagesPullingTime="50.571227962s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:40:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:41:42.810129133 +0000 UTC m=+244.987617751" watchObservedRunningTime="2026-10-30 21:41:42.811222591 +0000 UTC m=+244.988711207"
Oct 30 21:41:42.884893 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:42.884867    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cghp2w"
Oct 30 21:41:42.924048 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:42.923950    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agzz78" podStartSLOduration=3.429385867 podStartE2EDuration="52.923935213s" totalImagesPullingTime="49.494549346s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:40:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:41:42.922718732 +0000 UTC m=+245.100207351" watchObservedRunningTime="2026-10-30 21:41:42.923935213 +0000 UTC m=+245.101423821"
Oct 30 21:41:42.942002 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:42.941976    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2-bundle\" (UniqueName: \"kubernetes.io/empty-dir/d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2-bundle\") pod \"d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2\" (UID: \"d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2\") "
Oct 30 21:41:42.942140 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:42.942024    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2-kube-api-access-b5qvf\" (UniqueName: \"kubernetes.io/projected/d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2-kube-api-access-b5qvf\") pod \"d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2\" (UID: \"d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2\") "
Oct 30 21:41:42.942140 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:42.942071    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2-util\" (UniqueName: \"kubernetes.io/empty-dir/d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2-util\") pod \"d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2\" (UID: \"d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2\") "
Oct 30 21:41:42.942581 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:42.942556    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2-bundle" pod "d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2" (UID: "d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:42.944210 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:42.944176    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2-kube-api-access-b5qvf" pod "d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2" (UID: "d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2"). InnerVolumeSpecName "kube-api-access-b5qvf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:41:42.948113 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:42.948089    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2-util" pod "d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2" (UID: "d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:43.043724 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:43.043690    4871 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2-util\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:43.043724 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:43.043718    4871 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2-bundle\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:43.043998 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:43.043735    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5qvf\" (UniqueName: \"kubernetes.io/projected/d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2-kube-api-access-b5qvf\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:43.715874 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:43.715838    4871 generic.go:356] "Generic (PLEG): container finished" podID="9103c07d-5a90-4736-a6a6-d6f874fc1152" containerID="a225db357f148e71b74e37f828da5132620c8be9c2b17f8f9c55c8bf4bc3562c" exitCode=0
Oct 30 21:41:43.716340 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:43.715924    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axbv8k" event={"ID":"9103c07d-5a90-4736-a6a6-d6f874fc1152","Type":"ContainerDied","Data":"a225db357f148e71b74e37f828da5132620c8be9c2b17f8f9c55c8bf4bc3562c"}
Oct 30 21:41:43.718356 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:43.718322    4871 generic.go:356] "Generic (PLEG): container finished" podID="cfdfd8e2-c269-41a5-847f-2760cd6577c7" containerID="c6d6dc197ec7513ab6d59ccdfa35dce2cf8c299d64f2c5d6286741a03aa2a1e0" exitCode=0
Oct 30 21:41:43.718471 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:43.718393    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agzz78" event={"ID":"cfdfd8e2-c269-41a5-847f-2760cd6577c7","Type":"ContainerDied","Data":"c6d6dc197ec7513ab6d59ccdfa35dce2cf8c299d64f2c5d6286741a03aa2a1e0"}
Oct 30 21:41:43.720858 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:43.720828    4871 generic.go:356] "Generic (PLEG): container finished" podID="53353129-6959-43a8-83c5-e845c7869516" containerID="2c51e8430ba0018472c983a0acd948e23dc8920d432af9cbc819442a109ea108" exitCode=0
Oct 30 21:41:43.720974 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:43.720911    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vsdmv" event={"ID":"53353129-6959-43a8-83c5-e845c7869516","Type":"ContainerDied","Data":"2c51e8430ba0018472c983a0acd948e23dc8920d432af9cbc819442a109ea108"}
Oct 30 21:41:43.722980 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:43.722960    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cghp2w"
Oct 30 21:41:43.723123 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:43.722959    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cghp2w" event={"ID":"d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2","Type":"ContainerDied","Data":"de075a6196aac0aa85546890f3153e4d986e1011b5c94381b5af6ac7b7174832"}
Oct 30 21:41:43.723123 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:43.723076    4871 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="de075a6196aac0aa85546890f3153e4d986e1011b5c94381b5af6ac7b7174832"
Oct 30 21:41:43.725141 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:43.725119    4871 generic.go:356] "Generic (PLEG): container finished" podID="036a0620-585b-4e3f-bb7e-5f4bb7799498" containerID="d424df9957a58a05ff4cb1cad402a6232f6118500f567495182337f18d69cc33" exitCode=0
Oct 30 21:41:43.725280 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:43.725195    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fjfzw" event={"ID":"036a0620-585b-4e3f-bb7e-5f4bb7799498","Type":"ContainerDied","Data":"d424df9957a58a05ff4cb1cad402a6232f6118500f567495182337f18d69cc33"}
Oct 30 21:41:43.727337 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:43.727309    4871 generic.go:356] "Generic (PLEG): container finished" podID="689c9fec-d928-483c-aea2-d5cd577213ff" containerID="ab201b45215d10adfb83cec94149783ea01eb538d95129f639dee944226b3d18" exitCode=0
Oct 30 21:41:43.727429 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:43.727379    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqplt6" event={"ID":"689c9fec-d928-483c-aea2-d5cd577213ff","Type":"ContainerDied","Data":"ab201b45215d10adfb83cec94149783ea01eb538d95129f639dee944226b3d18"}
Oct 30 21:41:43.867120 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:43.867087    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6b9qt6"
Oct 30 21:41:43.910220 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:43.910197    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x5g46"
Oct 30 21:41:43.913205 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:43.913185    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gcgr7"
Oct 30 21:41:43.950740 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:43.950704    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51e4633e-d58b-4843-92e1-35c0bfb629d3-util\" (UniqueName: \"kubernetes.io/empty-dir/51e4633e-d58b-4843-92e1-35c0bfb629d3-util\") pod \"51e4633e-d58b-4843-92e1-35c0bfb629d3\" (UID: \"51e4633e-d58b-4843-92e1-35c0bfb629d3\") "
Oct 30 21:41:43.950916 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:43.950772    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c82c2960-1446-4516-bbe9-dc7886be330d-kube-api-access-d8qr2\" (UniqueName: \"kubernetes.io/projected/c82c2960-1446-4516-bbe9-dc7886be330d-kube-api-access-d8qr2\") pod \"c82c2960-1446-4516-bbe9-dc7886be330d\" (UID: \"c82c2960-1446-4516-bbe9-dc7886be330d\") "
Oct 30 21:41:43.950916 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:43.950802    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c82c2960-1446-4516-bbe9-dc7886be330d-util\" (UniqueName: \"kubernetes.io/empty-dir/c82c2960-1446-4516-bbe9-dc7886be330d-util\") pod \"c82c2960-1446-4516-bbe9-dc7886be330d\" (UID: \"c82c2960-1446-4516-bbe9-dc7886be330d\") "
Oct 30 21:41:43.950916 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:43.950822    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30065d64-eb7a-485a-97d6-9997da5c70fd-util\" (UniqueName: \"kubernetes.io/empty-dir/30065d64-eb7a-485a-97d6-9997da5c70fd-util\") pod \"30065d64-eb7a-485a-97d6-9997da5c70fd\" (UID: \"30065d64-eb7a-485a-97d6-9997da5c70fd\") "
Oct 30 21:41:43.950916 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:43.950848    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/51e4633e-d58b-4843-92e1-35c0bfb629d3-kube-api-access-9cqks\" (UniqueName: \"kubernetes.io/projected/51e4633e-d58b-4843-92e1-35c0bfb629d3-kube-api-access-9cqks\") pod \"51e4633e-d58b-4843-92e1-35c0bfb629d3\" (UID: \"51e4633e-d58b-4843-92e1-35c0bfb629d3\") "
Oct 30 21:41:43.950916 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:43.950888    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c82c2960-1446-4516-bbe9-dc7886be330d-bundle\" (UniqueName: \"kubernetes.io/empty-dir/c82c2960-1446-4516-bbe9-dc7886be330d-bundle\") pod \"c82c2960-1446-4516-bbe9-dc7886be330d\" (UID: \"c82c2960-1446-4516-bbe9-dc7886be330d\") "
Oct 30 21:41:43.951183 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:43.950929    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30065d64-eb7a-485a-97d6-9997da5c70fd-bundle\" (UniqueName: \"kubernetes.io/empty-dir/30065d64-eb7a-485a-97d6-9997da5c70fd-bundle\") pod \"30065d64-eb7a-485a-97d6-9997da5c70fd\" (UID: \"30065d64-eb7a-485a-97d6-9997da5c70fd\") "
Oct 30 21:41:43.951183 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:43.950976    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/30065d64-eb7a-485a-97d6-9997da5c70fd-kube-api-access-sb7hv\" (UniqueName: \"kubernetes.io/projected/30065d64-eb7a-485a-97d6-9997da5c70fd-kube-api-access-sb7hv\") pod \"30065d64-eb7a-485a-97d6-9997da5c70fd\" (UID: \"30065d64-eb7a-485a-97d6-9997da5c70fd\") "
Oct 30 21:41:43.951183 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:43.951011    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51e4633e-d58b-4843-92e1-35c0bfb629d3-bundle\" (UniqueName: \"kubernetes.io/empty-dir/51e4633e-d58b-4843-92e1-35c0bfb629d3-bundle\") pod \"51e4633e-d58b-4843-92e1-35c0bfb629d3\" (UID: \"51e4633e-d58b-4843-92e1-35c0bfb629d3\") "
Oct 30 21:41:43.951712 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:43.951678    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c82c2960-1446-4516-bbe9-dc7886be330d-bundle" pod "c82c2960-1446-4516-bbe9-dc7886be330d" (UID: "c82c2960-1446-4516-bbe9-dc7886be330d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:43.951801 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:43.951693    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51e4633e-d58b-4843-92e1-35c0bfb629d3-bundle" pod "51e4633e-d58b-4843-92e1-35c0bfb629d3" (UID: "51e4633e-d58b-4843-92e1-35c0bfb629d3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:43.952198 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:43.952173    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30065d64-eb7a-485a-97d6-9997da5c70fd-bundle" pod "30065d64-eb7a-485a-97d6-9997da5c70fd" (UID: "30065d64-eb7a-485a-97d6-9997da5c70fd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:43.953535 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:43.953425    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51e4633e-d58b-4843-92e1-35c0bfb629d3-kube-api-access-9cqks" pod "51e4633e-d58b-4843-92e1-35c0bfb629d3" (UID: "51e4633e-d58b-4843-92e1-35c0bfb629d3"). InnerVolumeSpecName "kube-api-access-9cqks". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:41:43.953535 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:43.953497    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c82c2960-1446-4516-bbe9-dc7886be330d-kube-api-access-d8qr2" pod "c82c2960-1446-4516-bbe9-dc7886be330d" (UID: "c82c2960-1446-4516-bbe9-dc7886be330d"). InnerVolumeSpecName "kube-api-access-d8qr2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:41:43.953884 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:43.953864    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30065d64-eb7a-485a-97d6-9997da5c70fd-kube-api-access-sb7hv" pod "30065d64-eb7a-485a-97d6-9997da5c70fd" (UID: "30065d64-eb7a-485a-97d6-9997da5c70fd"). InnerVolumeSpecName "kube-api-access-sb7hv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:41:43.959645 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:43.959616    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30065d64-eb7a-485a-97d6-9997da5c70fd-util" pod "30065d64-eb7a-485a-97d6-9997da5c70fd" (UID: "30065d64-eb7a-485a-97d6-9997da5c70fd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:43.959769 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:43.959749    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51e4633e-d58b-4843-92e1-35c0bfb629d3-util" pod "51e4633e-d58b-4843-92e1-35c0bfb629d3" (UID: "51e4633e-d58b-4843-92e1-35c0bfb629d3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:43.961634 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:43.961607    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c82c2960-1446-4516-bbe9-dc7886be330d-util" pod "c82c2960-1446-4516-bbe9-dc7886be330d" (UID: "c82c2960-1446-4516-bbe9-dc7886be330d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:44.052521 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:44.052485    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8qr2\" (UniqueName: \"kubernetes.io/projected/c82c2960-1446-4516-bbe9-dc7886be330d-kube-api-access-d8qr2\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:44.052521 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:44.052515    4871 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c82c2960-1446-4516-bbe9-dc7886be330d-util\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:44.052521 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:44.052524    4871 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/30065d64-eb7a-485a-97d6-9997da5c70fd-util\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:44.052521 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:44.052533    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9cqks\" (UniqueName: \"kubernetes.io/projected/51e4633e-d58b-4843-92e1-35c0bfb629d3-kube-api-access-9cqks\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:44.052806 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:44.052541    4871 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c82c2960-1446-4516-bbe9-dc7886be330d-bundle\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:44.052806 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:44.052550    4871 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/30065d64-eb7a-485a-97d6-9997da5c70fd-bundle\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:44.052806 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:44.052559    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sb7hv\" (UniqueName: \"kubernetes.io/projected/30065d64-eb7a-485a-97d6-9997da5c70fd-kube-api-access-sb7hv\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:44.052806 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:44.052567    4871 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/51e4633e-d58b-4843-92e1-35c0bfb629d3-bundle\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:44.052806 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:44.052575    4871 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/51e4633e-d58b-4843-92e1-35c0bfb629d3-util\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:44.734192 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:44.734160    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6b9qt6"
Oct 30 21:41:44.734661 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:44.734171    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6b9qt6" event={"ID":"51e4633e-d58b-4843-92e1-35c0bfb629d3","Type":"ContainerDied","Data":"0c8817db71004561adddd47366b67fadb44c367b344adf13aae8c87bf62ee50d"}
Oct 30 21:41:44.734661 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:44.734321    4871 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0c8817db71004561adddd47366b67fadb44c367b344adf13aae8c87bf62ee50d"
Oct 30 21:41:44.736462 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:44.736433    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x5g46" event={"ID":"30065d64-eb7a-485a-97d6-9997da5c70fd","Type":"ContainerDied","Data":"ce32efbc0cda2caaedddd1f11062b4fd3b22c7faecb01966e592bf24d97ac445"}
Oct 30 21:41:44.736586 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:44.736484    4871 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ce32efbc0cda2caaedddd1f11062b4fd3b22c7faecb01966e592bf24d97ac445"
Oct 30 21:41:44.736586 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:44.736496    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x5g46"
Oct 30 21:41:44.739053 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:44.739031    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gcgr7"
Oct 30 21:41:44.739147 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:44.739040    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gcgr7" event={"ID":"c82c2960-1446-4516-bbe9-dc7886be330d","Type":"ContainerDied","Data":"8aeca314aac99557d7a470b1aa0c7a5e0ed121009176e79159f03db5cddcd9ea"}
Oct 30 21:41:44.739147 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:44.739087    4871 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8aeca314aac99557d7a470b1aa0c7a5e0ed121009176e79159f03db5cddcd9ea"
Oct 30 21:41:44.931270 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:44.929874    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axbv8k"
Oct 30 21:41:44.944866 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:44.944831    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fjfzw"
Oct 30 21:41:44.958369 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:44.958336    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqplt6"
Oct 30 21:41:44.970227 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:44.970157    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agzz78"
Oct 30 21:41:44.970227 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:44.970213    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/036a0620-585b-4e3f-bb7e-5f4bb7799498-util\" (UniqueName: \"kubernetes.io/empty-dir/036a0620-585b-4e3f-bb7e-5f4bb7799498-util\") pod \"036a0620-585b-4e3f-bb7e-5f4bb7799498\" (UID: \"036a0620-585b-4e3f-bb7e-5f4bb7799498\") "
Oct 30 21:41:44.970420 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:44.970267    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9103c07d-5a90-4736-a6a6-d6f874fc1152-kube-api-access-scdsx\" (UniqueName: \"kubernetes.io/projected/9103c07d-5a90-4736-a6a6-d6f874fc1152-kube-api-access-scdsx\") pod \"9103c07d-5a90-4736-a6a6-d6f874fc1152\" (UID: \"9103c07d-5a90-4736-a6a6-d6f874fc1152\") "
Oct 30 21:41:44.970420 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:44.970309    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/036a0620-585b-4e3f-bb7e-5f4bb7799498-kube-api-access-zc5cx\" (UniqueName: \"kubernetes.io/projected/036a0620-585b-4e3f-bb7e-5f4bb7799498-kube-api-access-zc5cx\") pod \"036a0620-585b-4e3f-bb7e-5f4bb7799498\" (UID: \"036a0620-585b-4e3f-bb7e-5f4bb7799498\") "
Oct 30 21:41:44.970420 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:44.970332    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/036a0620-585b-4e3f-bb7e-5f4bb7799498-bundle\" (UniqueName: \"kubernetes.io/empty-dir/036a0620-585b-4e3f-bb7e-5f4bb7799498-bundle\") pod \"036a0620-585b-4e3f-bb7e-5f4bb7799498\" (UID: \"036a0620-585b-4e3f-bb7e-5f4bb7799498\") "
Oct 30 21:41:44.970420 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:44.970373    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9103c07d-5a90-4736-a6a6-d6f874fc1152-bundle\" (UniqueName: \"kubernetes.io/empty-dir/9103c07d-5a90-4736-a6a6-d6f874fc1152-bundle\") pod \"9103c07d-5a90-4736-a6a6-d6f874fc1152\" (UID: \"9103c07d-5a90-4736-a6a6-d6f874fc1152\") "
Oct 30 21:41:44.970603 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:44.970429    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9103c07d-5a90-4736-a6a6-d6f874fc1152-util\" (UniqueName: \"kubernetes.io/empty-dir/9103c07d-5a90-4736-a6a6-d6f874fc1152-util\") pod \"9103c07d-5a90-4736-a6a6-d6f874fc1152\" (UID: \"9103c07d-5a90-4736-a6a6-d6f874fc1152\") "
Oct 30 21:41:44.971035 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:44.970943    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/036a0620-585b-4e3f-bb7e-5f4bb7799498-bundle" pod "036a0620-585b-4e3f-bb7e-5f4bb7799498" (UID: "036a0620-585b-4e3f-bb7e-5f4bb7799498"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:44.971184 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:44.971154    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9103c07d-5a90-4736-a6a6-d6f874fc1152-bundle" pod "9103c07d-5a90-4736-a6a6-d6f874fc1152" (UID: "9103c07d-5a90-4736-a6a6-d6f874fc1152"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:44.972673 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:44.972648    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9103c07d-5a90-4736-a6a6-d6f874fc1152-kube-api-access-scdsx" pod "9103c07d-5a90-4736-a6a6-d6f874fc1152" (UID: "9103c07d-5a90-4736-a6a6-d6f874fc1152"). InnerVolumeSpecName "kube-api-access-scdsx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:41:44.973305 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:44.973284    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/036a0620-585b-4e3f-bb7e-5f4bb7799498-kube-api-access-zc5cx" pod "036a0620-585b-4e3f-bb7e-5f4bb7799498" (UID: "036a0620-585b-4e3f-bb7e-5f4bb7799498"). InnerVolumeSpecName "kube-api-access-zc5cx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:41:44.974452 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:44.974435    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vsdmv"
Oct 30 21:41:44.976624 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:44.976604    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/036a0620-585b-4e3f-bb7e-5f4bb7799498-util" pod "036a0620-585b-4e3f-bb7e-5f4bb7799498" (UID: "036a0620-585b-4e3f-bb7e-5f4bb7799498"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:44.977278 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:44.977256    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9103c07d-5a90-4736-a6a6-d6f874fc1152-util" pod "9103c07d-5a90-4736-a6a6-d6f874fc1152" (UID: "9103c07d-5a90-4736-a6a6-d6f874fc1152"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:45.071199 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.071157    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/689c9fec-d928-483c-aea2-d5cd577213ff-kube-api-access-zxmbn\" (UniqueName: \"kubernetes.io/projected/689c9fec-d928-483c-aea2-d5cd577213ff-kube-api-access-zxmbn\") pod \"689c9fec-d928-483c-aea2-d5cd577213ff\" (UID: \"689c9fec-d928-483c-aea2-d5cd577213ff\") "
Oct 30 21:41:45.071406 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.071263    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/689c9fec-d928-483c-aea2-d5cd577213ff-bundle\" (UniqueName: \"kubernetes.io/empty-dir/689c9fec-d928-483c-aea2-d5cd577213ff-bundle\") pod \"689c9fec-d928-483c-aea2-d5cd577213ff\" (UID: \"689c9fec-d928-483c-aea2-d5cd577213ff\") "
Oct 30 21:41:45.071406 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.071286    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cfdfd8e2-c269-41a5-847f-2760cd6577c7-kube-api-access-2dxxf\" (UniqueName: \"kubernetes.io/projected/cfdfd8e2-c269-41a5-847f-2760cd6577c7-kube-api-access-2dxxf\") pod \"cfdfd8e2-c269-41a5-847f-2760cd6577c7\" (UID: \"cfdfd8e2-c269-41a5-847f-2760cd6577c7\") "
Oct 30 21:41:45.071406 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.071318    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfdfd8e2-c269-41a5-847f-2760cd6577c7-bundle\" (UniqueName: \"kubernetes.io/empty-dir/cfdfd8e2-c269-41a5-847f-2760cd6577c7-bundle\") pod \"cfdfd8e2-c269-41a5-847f-2760cd6577c7\" (UID: \"cfdfd8e2-c269-41a5-847f-2760cd6577c7\") "
Oct 30 21:41:45.071406 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.071339    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfdfd8e2-c269-41a5-847f-2760cd6577c7-util\" (UniqueName: \"kubernetes.io/empty-dir/cfdfd8e2-c269-41a5-847f-2760cd6577c7-util\") pod \"cfdfd8e2-c269-41a5-847f-2760cd6577c7\" (UID: \"cfdfd8e2-c269-41a5-847f-2760cd6577c7\") "
Oct 30 21:41:45.071406 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.071364    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/53353129-6959-43a8-83c5-e845c7869516-bundle\" (UniqueName: \"kubernetes.io/empty-dir/53353129-6959-43a8-83c5-e845c7869516-bundle\") pod \"53353129-6959-43a8-83c5-e845c7869516\" (UID: \"53353129-6959-43a8-83c5-e845c7869516\") "
Oct 30 21:41:45.071406 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.071390    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/689c9fec-d928-483c-aea2-d5cd577213ff-util\" (UniqueName: \"kubernetes.io/empty-dir/689c9fec-d928-483c-aea2-d5cd577213ff-util\") pod \"689c9fec-d928-483c-aea2-d5cd577213ff\" (UID: \"689c9fec-d928-483c-aea2-d5cd577213ff\") "
Oct 30 21:41:45.071715 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.071420    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/53353129-6959-43a8-83c5-e845c7869516-kube-api-access-ndv95\" (UniqueName: \"kubernetes.io/projected/53353129-6959-43a8-83c5-e845c7869516-kube-api-access-ndv95\") pod \"53353129-6959-43a8-83c5-e845c7869516\" (UID: \"53353129-6959-43a8-83c5-e845c7869516\") "
Oct 30 21:41:45.071715 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.071453    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/53353129-6959-43a8-83c5-e845c7869516-util\" (UniqueName: \"kubernetes.io/empty-dir/53353129-6959-43a8-83c5-e845c7869516-util\") pod \"53353129-6959-43a8-83c5-e845c7869516\" (UID: \"53353129-6959-43a8-83c5-e845c7869516\") "
Oct 30 21:41:45.071715 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.071672    4871 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9103c07d-5a90-4736-a6a6-d6f874fc1152-util\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:45.071715 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.071691    4871 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/036a0620-585b-4e3f-bb7e-5f4bb7799498-util\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:45.071715 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.071707    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-scdsx\" (UniqueName: \"kubernetes.io/projected/9103c07d-5a90-4736-a6a6-d6f874fc1152-kube-api-access-scdsx\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:45.071948 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.071721    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zc5cx\" (UniqueName: \"kubernetes.io/projected/036a0620-585b-4e3f-bb7e-5f4bb7799498-kube-api-access-zc5cx\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:45.071948 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.071734    4871 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/036a0620-585b-4e3f-bb7e-5f4bb7799498-bundle\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:45.071948 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.071747    4871 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9103c07d-5a90-4736-a6a6-d6f874fc1152-bundle\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:45.072315 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.072279    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/689c9fec-d928-483c-aea2-d5cd577213ff-bundle" pod "689c9fec-d928-483c-aea2-d5cd577213ff" (UID: "689c9fec-d928-483c-aea2-d5cd577213ff"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:45.073178 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.072467    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53353129-6959-43a8-83c5-e845c7869516-bundle" pod "53353129-6959-43a8-83c5-e845c7869516" (UID: "53353129-6959-43a8-83c5-e845c7869516"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:45.073178 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.072531    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfdfd8e2-c269-41a5-847f-2760cd6577c7-bundle" pod "cfdfd8e2-c269-41a5-847f-2760cd6577c7" (UID: "cfdfd8e2-c269-41a5-847f-2760cd6577c7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:45.073850 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.073819    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfdfd8e2-c269-41a5-847f-2760cd6577c7-kube-api-access-2dxxf" pod "cfdfd8e2-c269-41a5-847f-2760cd6577c7" (UID: "cfdfd8e2-c269-41a5-847f-2760cd6577c7"). InnerVolumeSpecName "kube-api-access-2dxxf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:41:45.074005 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.073983    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/689c9fec-d928-483c-aea2-d5cd577213ff-kube-api-access-zxmbn" pod "689c9fec-d928-483c-aea2-d5cd577213ff" (UID: "689c9fec-d928-483c-aea2-d5cd577213ff"). InnerVolumeSpecName "kube-api-access-zxmbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:41:45.074468 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.074436    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53353129-6959-43a8-83c5-e845c7869516-kube-api-access-ndv95" pod "53353129-6959-43a8-83c5-e845c7869516" (UID: "53353129-6959-43a8-83c5-e845c7869516"). InnerVolumeSpecName "kube-api-access-ndv95". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:41:45.077976 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.077954    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfdfd8e2-c269-41a5-847f-2760cd6577c7-util" pod "cfdfd8e2-c269-41a5-847f-2760cd6577c7" (UID: "cfdfd8e2-c269-41a5-847f-2760cd6577c7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:45.079174 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.079155    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53353129-6959-43a8-83c5-e845c7869516-util" pod "53353129-6959-43a8-83c5-e845c7869516" (UID: "53353129-6959-43a8-83c5-e845c7869516"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:45.079468 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.079449    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/689c9fec-d928-483c-aea2-d5cd577213ff-util" pod "689c9fec-d928-483c-aea2-d5cd577213ff" (UID: "689c9fec-d928-483c-aea2-d5cd577213ff"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:45.172383 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.172342    4871 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cfdfd8e2-c269-41a5-847f-2760cd6577c7-bundle\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:45.172383 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.172373    4871 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cfdfd8e2-c269-41a5-847f-2760cd6577c7-util\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:45.172383 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.172382    4871 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/53353129-6959-43a8-83c5-e845c7869516-bundle\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:45.172383 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.172391    4871 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/689c9fec-d928-483c-aea2-d5cd577213ff-util\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:45.172638 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.172403    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndv95\" (UniqueName: \"kubernetes.io/projected/53353129-6959-43a8-83c5-e845c7869516-kube-api-access-ndv95\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:45.172638 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.172412    4871 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/53353129-6959-43a8-83c5-e845c7869516-util\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:45.172638 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.172422    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zxmbn\" (UniqueName: \"kubernetes.io/projected/689c9fec-d928-483c-aea2-d5cd577213ff-kube-api-access-zxmbn\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:45.172638 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.172430    4871 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/689c9fec-d928-483c-aea2-d5cd577213ff-bundle\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:45.172638 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.172439    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dxxf\" (UniqueName: \"kubernetes.io/projected/cfdfd8e2-c269-41a5-847f-2760cd6577c7-kube-api-access-2dxxf\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:45.745486 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.745449    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqplt6"
Oct 30 21:41:45.745922 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.745438    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqplt6" event={"ID":"689c9fec-d928-483c-aea2-d5cd577213ff","Type":"ContainerDied","Data":"fc1e0f562eef1f0156656a4aace70dfc49c46e6d043feb01423abe03437e5180"}
Oct 30 21:41:45.745922 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.745607    4871 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fc1e0f562eef1f0156656a4aace70dfc49c46e6d043feb01423abe03437e5180"
Oct 30 21:41:45.747588 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.747548    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axbv8k" event={"ID":"9103c07d-5a90-4736-a6a6-d6f874fc1152","Type":"ContainerDied","Data":"306708b6cda8f6ebf61916b8a811b3dac8d9b08a1e4bb995b01f32c82205f7c2"}
Oct 30 21:41:45.747588 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.747572    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847axbv8k"
Oct 30 21:41:45.747773 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.747605    4871 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="306708b6cda8f6ebf61916b8a811b3dac8d9b08a1e4bb995b01f32c82205f7c2"
Oct 30 21:41:45.749729 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.749701    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agzz78" event={"ID":"cfdfd8e2-c269-41a5-847f-2760cd6577c7","Type":"ContainerDied","Data":"ed43f16e0318efeb525011bfd73652a581961a9dfd32088049eeb40a5e729fde"}
Oct 30 21:41:45.749847 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.749738    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agzz78"
Oct 30 21:41:45.749847 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.749749    4871 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ed43f16e0318efeb525011bfd73652a581961a9dfd32088049eeb40a5e729fde"
Oct 30 21:41:45.751755 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.751733    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vsdmv"
Oct 30 21:41:45.751755 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.751746    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vsdmv" event={"ID":"53353129-6959-43a8-83c5-e845c7869516","Type":"ContainerDied","Data":"5bd7bc6e7d0f739a857e2bd3cd2bf1182dd947db6c27bfb4861d26d187476407"}
Oct 30 21:41:45.751890 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.751788    4871 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5bd7bc6e7d0f739a857e2bd3cd2bf1182dd947db6c27bfb4861d26d187476407"
Oct 30 21:41:45.754304 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.754283    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fjfzw" event={"ID":"036a0620-585b-4e3f-bb7e-5f4bb7799498","Type":"ContainerDied","Data":"7c42e83147ad4f90a4c7599fab9979b2117e26dbc729cc78439c18f0dbe9af37"}
Oct 30 21:41:45.754406 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.754321    4871 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7c42e83147ad4f90a4c7599fab9979b2117e26dbc729cc78439c18f0dbe9af37"
Oct 30 21:41:45.754406 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:45.754355    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fjfzw"
Oct 30 21:41:46.051197 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051111    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71cwddk"]
Oct 30 21:41:46.051431 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051419    4871 cpu_manager.go:414] "containerMap: removing container" podUID="53353129-6959-43a8-83c5-e845c7869516" containerName="util"
Oct 30 21:41:46.051478 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051433    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="53353129-6959-43a8-83c5-e845c7869516" containerName="util"
Oct 30 21:41:46.051478 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051441    4871 cpu_manager.go:414] "containerMap: removing container" podUID="8d401654-8a4c-4a79-b2d3-612765312421" containerName="extract"
Oct 30 21:41:46.051478 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051446    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8d401654-8a4c-4a79-b2d3-612765312421" containerName="extract"
Oct 30 21:41:46.051478 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051454    4871 cpu_manager.go:414] "containerMap: removing container" podUID="c82c2960-1446-4516-bbe9-dc7886be330d" containerName="extract"
Oct 30 21:41:46.051478 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051459    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c82c2960-1446-4516-bbe9-dc7886be330d" containerName="extract"
Oct 30 21:41:46.051478 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051467    4871 cpu_manager.go:414] "containerMap: removing container" podUID="689c9fec-d928-483c-aea2-d5cd577213ff" containerName="pull"
Oct 30 21:41:46.051478 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051471    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="689c9fec-d928-483c-aea2-d5cd577213ff" containerName="pull"
Oct 30 21:41:46.051478 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051478    4871 cpu_manager.go:414] "containerMap: removing container" podUID="53353129-6959-43a8-83c5-e845c7869516" containerName="pull"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051483    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="53353129-6959-43a8-83c5-e845c7869516" containerName="pull"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051489    4871 cpu_manager.go:414] "containerMap: removing container" podUID="51e4633e-d58b-4843-92e1-35c0bfb629d3" containerName="extract"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051494    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="51e4633e-d58b-4843-92e1-35c0bfb629d3" containerName="extract"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051503    4871 cpu_manager.go:414] "containerMap: removing container" podUID="036a0620-585b-4e3f-bb7e-5f4bb7799498" containerName="util"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051507    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="036a0620-585b-4e3f-bb7e-5f4bb7799498" containerName="util"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051514    4871 cpu_manager.go:414] "containerMap: removing container" podUID="d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2" containerName="pull"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051518    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2" containerName="pull"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051523    4871 cpu_manager.go:414] "containerMap: removing container" podUID="53353129-6959-43a8-83c5-e845c7869516" containerName="extract"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051529    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="53353129-6959-43a8-83c5-e845c7869516" containerName="extract"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051539    4871 cpu_manager.go:414] "containerMap: removing container" podUID="30065d64-eb7a-485a-97d6-9997da5c70fd" containerName="util"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051543    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="30065d64-eb7a-485a-97d6-9997da5c70fd" containerName="util"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051550    4871 cpu_manager.go:414] "containerMap: removing container" podUID="cfdfd8e2-c269-41a5-847f-2760cd6577c7" containerName="util"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051554    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cfdfd8e2-c269-41a5-847f-2760cd6577c7" containerName="util"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051560    4871 cpu_manager.go:414] "containerMap: removing container" podUID="d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2" containerName="util"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051564    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2" containerName="util"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051570    4871 cpu_manager.go:414] "containerMap: removing container" podUID="30065d64-eb7a-485a-97d6-9997da5c70fd" containerName="pull"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051575    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="30065d64-eb7a-485a-97d6-9997da5c70fd" containerName="pull"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051581    4871 cpu_manager.go:414] "containerMap: removing container" podUID="51e4633e-d58b-4843-92e1-35c0bfb629d3" containerName="pull"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051585    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="51e4633e-d58b-4843-92e1-35c0bfb629d3" containerName="pull"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051590    4871 cpu_manager.go:414] "containerMap: removing container" podUID="56a9d68e-fdea-4bd4-a96c-855199901791" containerName="extract"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051594    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="56a9d68e-fdea-4bd4-a96c-855199901791" containerName="extract"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051603    4871 cpu_manager.go:414] "containerMap: removing container" podUID="8d401654-8a4c-4a79-b2d3-612765312421" containerName="util"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051607    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8d401654-8a4c-4a79-b2d3-612765312421" containerName="util"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051614    4871 cpu_manager.go:414] "containerMap: removing container" podUID="c82c2960-1446-4516-bbe9-dc7886be330d" containerName="util"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051618    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c82c2960-1446-4516-bbe9-dc7886be330d" containerName="util"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051625    4871 cpu_manager.go:414] "containerMap: removing container" podUID="c82c2960-1446-4516-bbe9-dc7886be330d" containerName="pull"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051629    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c82c2960-1446-4516-bbe9-dc7886be330d" containerName="pull"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051637    4871 cpu_manager.go:414] "containerMap: removing container" podUID="9103c07d-5a90-4736-a6a6-d6f874fc1152" containerName="util"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051641    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9103c07d-5a90-4736-a6a6-d6f874fc1152" containerName="util"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051646    4871 cpu_manager.go:414] "containerMap: removing container" podUID="036a0620-585b-4e3f-bb7e-5f4bb7799498" containerName="extract"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051651    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="036a0620-585b-4e3f-bb7e-5f4bb7799498" containerName="extract"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051657    4871 cpu_manager.go:414] "containerMap: removing container" podUID="56a9d68e-fdea-4bd4-a96c-855199901791" containerName="util"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051663    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="56a9d68e-fdea-4bd4-a96c-855199901791" containerName="util"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051668    4871 cpu_manager.go:414] "containerMap: removing container" podUID="570a9217-1578-43e4-a5fd-65c8b261b535" containerName="extract"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051672    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="570a9217-1578-43e4-a5fd-65c8b261b535" containerName="extract"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051678    4871 cpu_manager.go:414] "containerMap: removing container" podUID="8d401654-8a4c-4a79-b2d3-612765312421" containerName="pull"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051682    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8d401654-8a4c-4a79-b2d3-612765312421" containerName="pull"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051688    4871 cpu_manager.go:414] "containerMap: removing container" podUID="cfdfd8e2-c269-41a5-847f-2760cd6577c7" containerName="pull"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051692    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cfdfd8e2-c269-41a5-847f-2760cd6577c7" containerName="pull"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051699    4871 cpu_manager.go:414] "containerMap: removing container" podUID="036a0620-585b-4e3f-bb7e-5f4bb7799498" containerName="pull"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051703    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="036a0620-585b-4e3f-bb7e-5f4bb7799498" containerName="pull"
Oct 30 21:41:46.051687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051708    4871 cpu_manager.go:414] "containerMap: removing container" podUID="30065d64-eb7a-485a-97d6-9997da5c70fd" containerName="extract"
Oct 30 21:41:46.052810 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051712    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="30065d64-eb7a-485a-97d6-9997da5c70fd" containerName="extract"
Oct 30 21:41:46.052810 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051718    4871 cpu_manager.go:414] "containerMap: removing container" podUID="cfdfd8e2-c269-41a5-847f-2760cd6577c7" containerName="extract"
Oct 30 21:41:46.052810 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051723    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cfdfd8e2-c269-41a5-847f-2760cd6577c7" containerName="extract"
Oct 30 21:41:46.052810 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051731    4871 cpu_manager.go:414] "containerMap: removing container" podUID="9103c07d-5a90-4736-a6a6-d6f874fc1152" containerName="pull"
Oct 30 21:41:46.052810 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051735    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9103c07d-5a90-4736-a6a6-d6f874fc1152" containerName="pull"
Oct 30 21:41:46.052810 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051744    4871 cpu_manager.go:414] "containerMap: removing container" podUID="689c9fec-d928-483c-aea2-d5cd577213ff" containerName="extract"
Oct 30 21:41:46.052810 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051748    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="689c9fec-d928-483c-aea2-d5cd577213ff" containerName="extract"
Oct 30 21:41:46.052810 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051755    4871 cpu_manager.go:414] "containerMap: removing container" podUID="570a9217-1578-43e4-a5fd-65c8b261b535" containerName="util"
Oct 30 21:41:46.052810 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051760    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="570a9217-1578-43e4-a5fd-65c8b261b535" containerName="util"
Oct 30 21:41:46.052810 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051766    4871 cpu_manager.go:414] "containerMap: removing container" podUID="689c9fec-d928-483c-aea2-d5cd577213ff" containerName="util"
Oct 30 21:41:46.052810 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051770    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="689c9fec-d928-483c-aea2-d5cd577213ff" containerName="util"
Oct 30 21:41:46.052810 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051777    4871 cpu_manager.go:414] "containerMap: removing container" podUID="570a9217-1578-43e4-a5fd-65c8b261b535" containerName="pull"
Oct 30 21:41:46.052810 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051782    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="570a9217-1578-43e4-a5fd-65c8b261b535" containerName="pull"
Oct 30 21:41:46.052810 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051787    4871 cpu_manager.go:414] "containerMap: removing container" podUID="d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2" containerName="extract"
Oct 30 21:41:46.052810 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051791    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2" containerName="extract"
Oct 30 21:41:46.052810 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051796    4871 cpu_manager.go:414] "containerMap: removing container" podUID="9103c07d-5a90-4736-a6a6-d6f874fc1152" containerName="extract"
Oct 30 21:41:46.052810 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051800    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9103c07d-5a90-4736-a6a6-d6f874fc1152" containerName="extract"
Oct 30 21:41:46.052810 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051806    4871 cpu_manager.go:414] "containerMap: removing container" podUID="56a9d68e-fdea-4bd4-a96c-855199901791" containerName="pull"
Oct 30 21:41:46.052810 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051811    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="56a9d68e-fdea-4bd4-a96c-855199901791" containerName="pull"
Oct 30 21:41:46.052810 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051817    4871 cpu_manager.go:414] "containerMap: removing container" podUID="51e4633e-d58b-4843-92e1-35c0bfb629d3" containerName="util"
Oct 30 21:41:46.052810 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051821    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="51e4633e-d58b-4843-92e1-35c0bfb629d3" containerName="util"
Oct 30 21:41:46.052810 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051867    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="689c9fec-d928-483c-aea2-d5cd577213ff" containerName="extract"
Oct 30 21:41:46.052810 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051874    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="d52b8b18-2a5c-43b0-8473-aa55cf6bd4c2" containerName="extract"
Oct 30 21:41:46.052810 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051880    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="9103c07d-5a90-4736-a6a6-d6f874fc1152" containerName="extract"
Oct 30 21:41:46.052810 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051886    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="8d401654-8a4c-4a79-b2d3-612765312421" containerName="extract"
Oct 30 21:41:46.052810 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051891    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="c82c2960-1446-4516-bbe9-dc7886be330d" containerName="extract"
Oct 30 21:41:46.052810 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051896    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="51e4633e-d58b-4843-92e1-35c0bfb629d3" containerName="extract"
Oct 30 21:41:46.052810 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051902    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="53353129-6959-43a8-83c5-e845c7869516" containerName="extract"
Oct 30 21:41:46.052810 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051907    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="56a9d68e-fdea-4bd4-a96c-855199901791" containerName="extract"
Oct 30 21:41:46.052810 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051913    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="036a0620-585b-4e3f-bb7e-5f4bb7799498" containerName="extract"
Oct 30 21:41:46.052810 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051919    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="570a9217-1578-43e4-a5fd-65c8b261b535" containerName="extract"
Oct 30 21:41:46.052810 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051925    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="30065d64-eb7a-485a-97d6-9997da5c70fd" containerName="extract"
Oct 30 21:41:46.052810 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.051929    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="cfdfd8e2-c269-41a5-847f-2760cd6577c7" containerName="extract"
Oct 30 21:41:46.054845 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.054827    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71cwddk"
Oct 30 21:41:46.079371 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.079337    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbw6z\" (UniqueName: \"kubernetes.io/projected/a3069c63-8c6f-4226-b890-f779c66008ea-kube-api-access-kbw6z\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71cwddk\" (UID: \"a3069c63-8c6f-4226-b890-f779c66008ea\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71cwddk"
Oct 30 21:41:46.079560 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.079381    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3069c63-8c6f-4226-b890-f779c66008ea-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71cwddk\" (UID: \"a3069c63-8c6f-4226-b890-f779c66008ea\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71cwddk"
Oct 30 21:41:46.079560 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.079411    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3069c63-8c6f-4226-b890-f779c66008ea-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71cwddk\" (UID: \"a3069c63-8c6f-4226-b890-f779c66008ea\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71cwddk"
Oct 30 21:41:46.104361 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.104330    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 21:41:46.109198 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.109157    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 21:41:46.109774 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.109502    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e644421527c169d876fdb59235c927986d6bd2-9d2815fc-dockercfg-qgv9p\""
Oct 30 21:41:46.111839 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.111814    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71cwddk"]
Oct 30 21:41:46.180315 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.180281    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbw6z\" (UniqueName: \"kubernetes.io/projected/a3069c63-8c6f-4226-b890-f779c66008ea-kube-api-access-kbw6z\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71cwddk\" (UID: \"a3069c63-8c6f-4226-b890-f779c66008ea\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71cwddk"
Oct 30 21:41:46.180538 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.180329    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3069c63-8c6f-4226-b890-f779c66008ea-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71cwddk\" (UID: \"a3069c63-8c6f-4226-b890-f779c66008ea\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71cwddk"
Oct 30 21:41:46.180538 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.180366    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3069c63-8c6f-4226-b890-f779c66008ea-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71cwddk\" (UID: \"a3069c63-8c6f-4226-b890-f779c66008ea\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71cwddk"
Oct 30 21:41:46.180841 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.180816    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3069c63-8c6f-4226-b890-f779c66008ea-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71cwddk\" (UID: \"a3069c63-8c6f-4226-b890-f779c66008ea\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71cwddk"
Oct 30 21:41:46.180917 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.180830    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3069c63-8c6f-4226-b890-f779c66008ea-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71cwddk\" (UID: \"a3069c63-8c6f-4226-b890-f779c66008ea\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71cwddk"
Oct 30 21:41:46.238768 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.238725    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbw6z\" (UniqueName: \"kubernetes.io/projected/a3069c63-8c6f-4226-b890-f779c66008ea-kube-api-access-kbw6z\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71cwddk\" (UID: \"a3069c63-8c6f-4226-b890-f779c66008ea\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71cwddk"
Oct 30 21:41:46.367196 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.367164    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71cwddk"
Oct 30 21:41:46.598189 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.598139    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:46.738721 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.738695    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71cwddk"]
Oct 30 21:41:46.740975 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:41:46.740938    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3069c63_8c6f_4226_b890_f779c66008ea.slice/crio-060b0823bc0f26941327e4a4b376ad087bd2f6934eef956cd3408b5cf3f9c132 WatchSource:0}: Error finding container 060b0823bc0f26941327e4a4b376ad087bd2f6934eef956cd3408b5cf3f9c132: Status 404 returned error can't find the container with id 060b0823bc0f26941327e4a4b376ad087bd2f6934eef956cd3408b5cf3f9c132
Oct 30 21:41:46.762932 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:46.762903    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71cwddk" event={"ID":"a3069c63-8c6f-4226-b890-f779c66008ea","Type":"ContainerStarted","Data":"060b0823bc0f26941327e4a4b376ad087bd2f6934eef956cd3408b5cf3f9c132"}
Oct 30 21:41:47.768944 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:47.768902    4871 generic.go:356] "Generic (PLEG): container finished" podID="a3069c63-8c6f-4226-b890-f779c66008ea" containerID="0b2e2adfa07ae913a26c11188f8eccf0496f9c8bf39b72b96b361f4158d76d69" exitCode=0
Oct 30 21:41:47.769425 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:47.769012    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71cwddk" event={"ID":"a3069c63-8c6f-4226-b890-f779c66008ea","Type":"ContainerDied","Data":"0b2e2adfa07ae913a26c11188f8eccf0496f9c8bf39b72b96b361f4158d76d69"}
Oct 30 21:41:48.776259 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:48.776148    4871 generic.go:356] "Generic (PLEG): container finished" podID="a3069c63-8c6f-4226-b890-f779c66008ea" containerID="ef2196d8745730d39a89385c13ec423ed3678455dcd1b40c6455b388b2a678ed" exitCode=0
Oct 30 21:41:48.776259 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:48.776226    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71cwddk" event={"ID":"a3069c63-8c6f-4226-b890-f779c66008ea","Type":"ContainerDied","Data":"ef2196d8745730d39a89385c13ec423ed3678455dcd1b40c6455b388b2a678ed"}
Oct 30 21:41:49.229404 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:49.229365    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-b5k4x"]
Oct 30 21:41:49.231977 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:49.231958    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-b5k4x"
Oct 30 21:41:49.283250 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:49.283204    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Oct 30 21:41:49.283250 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:49.283232    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-sh87m\""
Oct 30 21:41:49.283468 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:49.283269    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:41:49.308751 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:49.308712    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdxg6\" (UniqueName: \"kubernetes.io/projected/7336f65c-e6d7-4649-af1b-4af03502194b-kube-api-access-zdxg6\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-b5k4x\" (UID: \"7336f65c-e6d7-4649-af1b-4af03502194b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-b5k4x"
Oct 30 21:41:49.308751 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:49.308754    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7336f65c-e6d7-4649-af1b-4af03502194b-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-b5k4x\" (UID: \"7336f65c-e6d7-4649-af1b-4af03502194b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-b5k4x"
Oct 30 21:41:49.329836 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:49.329806    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-b5k4x"]
Oct 30 21:41:49.409879 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:49.409839    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdxg6\" (UniqueName: \"kubernetes.io/projected/7336f65c-e6d7-4649-af1b-4af03502194b-kube-api-access-zdxg6\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-b5k4x\" (UID: \"7336f65c-e6d7-4649-af1b-4af03502194b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-b5k4x"
Oct 30 21:41:49.409879 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:49.409878    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7336f65c-e6d7-4649-af1b-4af03502194b-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-b5k4x\" (UID: \"7336f65c-e6d7-4649-af1b-4af03502194b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-b5k4x"
Oct 30 21:41:49.410276 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:49.410259    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7336f65c-e6d7-4649-af1b-4af03502194b-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-b5k4x\" (UID: \"7336f65c-e6d7-4649-af1b-4af03502194b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-b5k4x"
Oct 30 21:41:49.468107 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:49.468072    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdxg6\" (UniqueName: \"kubernetes.io/projected/7336f65c-e6d7-4649-af1b-4af03502194b-kube-api-access-zdxg6\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-b5k4x\" (UID: \"7336f65c-e6d7-4649-af1b-4af03502194b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-b5k4x"
Oct 30 21:41:49.543284 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:49.543179    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-b5k4x"
Oct 30 21:41:49.558903 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:49.558875    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:49.740834 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:49.740795    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:49.784090 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:49.784056    4871 generic.go:356] "Generic (PLEG): container finished" podID="a3069c63-8c6f-4226-b890-f779c66008ea" containerID="383606e87ee4d057a5b095e91f199313e619ec6774d2ad30f522242ac80ea88b" exitCode=0
Oct 30 21:41:49.784484 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:49.784117    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71cwddk" event={"ID":"a3069c63-8c6f-4226-b890-f779c66008ea","Type":"ContainerDied","Data":"383606e87ee4d057a5b095e91f199313e619ec6774d2ad30f522242ac80ea88b"}
Oct 30 21:41:49.962288 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:49.962224    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-b5k4x"]
Oct 30 21:41:49.965155 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:41:49.965127    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7336f65c_e6d7_4649_af1b_4af03502194b.slice/crio-5f9809db1c9a48a7189309219a2cffe36efcd0752b970ed0bd538520bb1e6ce2 WatchSource:0}: Error finding container 5f9809db1c9a48a7189309219a2cffe36efcd0752b970ed0bd538520bb1e6ce2: Status 404 returned error can't find the container with id 5f9809db1c9a48a7189309219a2cffe36efcd0752b970ed0bd538520bb1e6ce2
Oct 30 21:41:50.790837 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:50.790797    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-b5k4x" event={"ID":"7336f65c-e6d7-4649-af1b-4af03502194b","Type":"ContainerStarted","Data":"5f9809db1c9a48a7189309219a2cffe36efcd0752b970ed0bd538520bb1e6ce2"}
Oct 30 21:41:50.920212 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:50.920184    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71cwddk"
Oct 30 21:41:50.939024 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:50.938980    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:51.024355 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:51.024318    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3069c63-8c6f-4226-b890-f779c66008ea-bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3069c63-8c6f-4226-b890-f779c66008ea-bundle\") pod \"a3069c63-8c6f-4226-b890-f779c66008ea\" (UID: \"a3069c63-8c6f-4226-b890-f779c66008ea\") "
Oct 30 21:41:51.024557 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:51.024394    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3069c63-8c6f-4226-b890-f779c66008ea-util\" (UniqueName: \"kubernetes.io/empty-dir/a3069c63-8c6f-4226-b890-f779c66008ea-util\") pod \"a3069c63-8c6f-4226-b890-f779c66008ea\" (UID: \"a3069c63-8c6f-4226-b890-f779c66008ea\") "
Oct 30 21:41:51.024557 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:51.024421    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a3069c63-8c6f-4226-b890-f779c66008ea-kube-api-access-kbw6z\" (UniqueName: \"kubernetes.io/projected/a3069c63-8c6f-4226-b890-f779c66008ea-kube-api-access-kbw6z\") pod \"a3069c63-8c6f-4226-b890-f779c66008ea\" (UID: \"a3069c63-8c6f-4226-b890-f779c66008ea\") "
Oct 30 21:41:51.025022 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:51.024994    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3069c63-8c6f-4226-b890-f779c66008ea-bundle" pod "a3069c63-8c6f-4226-b890-f779c66008ea" (UID: "a3069c63-8c6f-4226-b890-f779c66008ea"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:51.026613 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:51.026592    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3069c63-8c6f-4226-b890-f779c66008ea-kube-api-access-kbw6z" pod "a3069c63-8c6f-4226-b890-f779c66008ea" (UID: "a3069c63-8c6f-4226-b890-f779c66008ea"). InnerVolumeSpecName "kube-api-access-kbw6z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:41:51.030706 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:51.030662    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3069c63-8c6f-4226-b890-f779c66008ea-util" pod "a3069c63-8c6f-4226-b890-f779c66008ea" (UID: "a3069c63-8c6f-4226-b890-f779c66008ea"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:51.125502 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:51.125459    4871 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3069c63-8c6f-4226-b890-f779c66008ea-util\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:51.125502 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:51.125503    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbw6z\" (UniqueName: \"kubernetes.io/projected/a3069c63-8c6f-4226-b890-f779c66008ea-kube-api-access-kbw6z\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:51.125706 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:51.125515    4871 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3069c63-8c6f-4226-b890-f779c66008ea-bundle\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:51.798065 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:51.798041    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71cwddk"
Oct 30 21:41:51.798481 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:51.798030    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71cwddk" event={"ID":"a3069c63-8c6f-4226-b890-f779c66008ea","Type":"ContainerDied","Data":"060b0823bc0f26941327e4a4b376ad087bd2f6934eef956cd3408b5cf3f9c132"}
Oct 30 21:41:51.798481 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:51.798191    4871 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="060b0823bc0f26941327e4a4b376ad087bd2f6934eef956cd3408b5cf3f9c132"
Oct 30 21:41:52.265177 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:52.265137    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:52.461509 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:52.461460    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:53.810125 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:53.810086    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-b5k4x" event={"ID":"7336f65c-e6d7-4649-af1b-4af03502194b","Type":"ContainerStarted","Data":"62e7bfc36f2d2b37d3bd9039b5b731e5eb94f77d11b5633510d512ea4a351094"}
Oct 30 21:41:54.023634 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:54.023570    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-b5k4x" podStartSLOduration=2.018698851 podStartE2EDuration="5.02355472s" totalImagesPullingTime="3.004855869s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:41:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:41:53.976375405 +0000 UTC m=+256.153864025" watchObservedRunningTime="2026-10-30 21:41:54.02355472 +0000 UTC m=+256.201043397"
Oct 30 21:41:55.120021 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:55.119982    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:55.315574 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:55.315539    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:57.336281 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:57.336180    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:57.876332 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:57.876293    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-tf9kq"]
Oct 30 21:41:57.876642 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:57.876630    4871 cpu_manager.go:414] "containerMap: removing container" podUID="a3069c63-8c6f-4226-b890-f779c66008ea" containerName="util"
Oct 30 21:41:57.876685 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:57.876643    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a3069c63-8c6f-4226-b890-f779c66008ea" containerName="util"
Oct 30 21:41:57.876685 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:57.876656    4871 cpu_manager.go:414] "containerMap: removing container" podUID="a3069c63-8c6f-4226-b890-f779c66008ea" containerName="extract"
Oct 30 21:41:57.876685 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:57.876661    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a3069c63-8c6f-4226-b890-f779c66008ea" containerName="extract"
Oct 30 21:41:57.876784 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:57.876689    4871 cpu_manager.go:414] "containerMap: removing container" podUID="a3069c63-8c6f-4226-b890-f779c66008ea" containerName="pull"
Oct 30 21:41:57.876784 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:57.876695    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a3069c63-8c6f-4226-b890-f779c66008ea" containerName="pull"
Oct 30 21:41:57.876784 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:57.876741    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="a3069c63-8c6f-4226-b890-f779c66008ea" containerName="extract"
Oct 30 21:41:57.879629 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:57.879613    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-tf9kq"
Oct 30 21:41:57.935757 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:57.935723    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-tf9kq"]
Oct 30 21:41:57.978360 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:57.978300    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Oct 30 21:41:57.978564 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:57.978390    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Oct 30 21:41:57.978564 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:57.978481    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-nnc9p\""
Oct 30 21:41:57.987653 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:57.987620    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fb5db419-4f7d-471c-b4eb-c5d1814b50b5-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-tf9kq\" (UID: \"fb5db419-4f7d-471c-b4eb-c5d1814b50b5\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-tf9kq"
Oct 30 21:41:57.987789 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:57.987745    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49mzh\" (UniqueName: \"kubernetes.io/projected/fb5db419-4f7d-471c-b4eb-c5d1814b50b5-kube-api-access-49mzh\") pod \"cert-manager-cainjector-5c57bbd4fc-tf9kq\" (UID: \"fb5db419-4f7d-471c-b4eb-c5d1814b50b5\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-tf9kq"
Oct 30 21:41:58.029195 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:58.029164    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:58.088882 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:58.088848    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49mzh\" (UniqueName: \"kubernetes.io/projected/fb5db419-4f7d-471c-b4eb-c5d1814b50b5-kube-api-access-49mzh\") pod \"cert-manager-cainjector-5c57bbd4fc-tf9kq\" (UID: \"fb5db419-4f7d-471c-b4eb-c5d1814b50b5\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-tf9kq"
Oct 30 21:41:58.089064 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:58.088907    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fb5db419-4f7d-471c-b4eb-c5d1814b50b5-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-tf9kq\" (UID: \"fb5db419-4f7d-471c-b4eb-c5d1814b50b5\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-tf9kq"
Oct 30 21:41:58.157023 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:58.156956    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49mzh\" (UniqueName: \"kubernetes.io/projected/fb5db419-4f7d-471c-b4eb-c5d1814b50b5-kube-api-access-49mzh\") pod \"cert-manager-cainjector-5c57bbd4fc-tf9kq\" (UID: \"fb5db419-4f7d-471c-b4eb-c5d1814b50b5\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-tf9kq"
Oct 30 21:41:58.157177 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:58.157035    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fb5db419-4f7d-471c-b4eb-c5d1814b50b5-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-tf9kq\" (UID: \"fb5db419-4f7d-471c-b4eb-c5d1814b50b5\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-tf9kq"
Oct 30 21:41:58.188437 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:58.188390    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:58.203694 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:58.203665    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-tf9kq"
Oct 30 21:41:58.578745 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:58.578715    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-tf9kq"]
Oct 30 21:41:58.580862 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:41:58.580833    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb5db419_4f7d_471c_b4eb_c5d1814b50b5.slice/crio-acf01ad6c34a8ec5d4664a7fcec8269b591e0a5bb5fe7bf4236366801e6eb209 WatchSource:0}: Error finding container acf01ad6c34a8ec5d4664a7fcec8269b591e0a5bb5fe7bf4236366801e6eb209: Status 404 returned error can't find the container with id acf01ad6c34a8ec5d4664a7fcec8269b591e0a5bb5fe7bf4236366801e6eb209
Oct 30 21:41:58.836595 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:58.836510    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-tf9kq" event={"ID":"fb5db419-4f7d-471c-b4eb-c5d1814b50b5","Type":"ContainerStarted","Data":"acf01ad6c34a8ec5d4664a7fcec8269b591e0a5bb5fe7bf4236366801e6eb209"}
Oct 30 21:41:59.389870 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:41:59.389833    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:02.208783 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:02.208752    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:02.362105 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:02.362074    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:02.859564 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:02.859470    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-tf9kq" event={"ID":"fb5db419-4f7d-471c-b4eb-c5d1814b50b5","Type":"ContainerStarted","Data":"8918e99699ba2adc2ae18c3eb63684ffc8e5d7ff20b4eede521aad1b6c0a87d1"}
Oct 30 21:42:02.969696 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:02.969638    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-tf9kq" podStartSLOduration=2.49476054 podStartE2EDuration="5.969621569s" totalImagesPullingTime="3.474861029s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:41:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:42:02.96775886 +0000 UTC m=+265.145247478" watchObservedRunningTime="2026-10-30 21:42:02.969621569 +0000 UTC m=+265.147110187"
Oct 30 21:42:03.489680 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:03.489644    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:03.642514 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:03.642475    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:03.937847 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:03.937809    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-vqwsv"]
Oct 30 21:42:03.940456 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:03.940440    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7b7bb4bc77-vqwsv"
Oct 30 21:42:03.990037 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:03.989993    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 30 21:42:04.034685 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:04.034643    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8r78\" (UniqueName: \"kubernetes.io/projected/b87c93be-5f5a-45ba-b820-ebd97c1a1c03-kube-api-access-d8r78\") pod \"authorino-operator-7b7bb4bc77-vqwsv\" (UID: \"b87c93be-5f5a-45ba-b820-ebd97c1a1c03\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-vqwsv"
Oct 30 21:42:04.036742 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:04.036722    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 30 21:42:04.036936 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:04.036915    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-94rj8\""
Oct 30 21:42:04.039212 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:04.039189    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-vqwsv"]
Oct 30 21:42:04.135929 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:04.135886    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8r78\" (UniqueName: \"kubernetes.io/projected/b87c93be-5f5a-45ba-b820-ebd97c1a1c03-kube-api-access-d8r78\") pod \"authorino-operator-7b7bb4bc77-vqwsv\" (UID: \"b87c93be-5f5a-45ba-b820-ebd97c1a1c03\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-vqwsv"
Oct 30 21:42:04.194559 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:04.194488    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8r78\" (UniqueName: \"kubernetes.io/projected/b87c93be-5f5a-45ba-b820-ebd97c1a1c03-kube-api-access-d8r78\") pod \"authorino-operator-7b7bb4bc77-vqwsv\" (UID: \"b87c93be-5f5a-45ba-b820-ebd97c1a1c03\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-vqwsv"
Oct 30 21:42:04.252611 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:04.252570    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7b7bb4bc77-vqwsv"
Oct 30 21:42:04.626260 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:04.626196    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-vqwsv"]
Oct 30 21:42:04.628494 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:42:04.628462    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb87c93be_5f5a_45ba_b820_ebd97c1a1c03.slice/crio-c6e984d8bf03dbd0ac0f943b5f78d77c1cd7aac76eb8570816c8bf4a85124e2b WatchSource:0}: Error finding container c6e984d8bf03dbd0ac0f943b5f78d77c1cd7aac76eb8570816c8bf4a85124e2b: Status 404 returned error can't find the container with id c6e984d8bf03dbd0ac0f943b5f78d77c1cd7aac76eb8570816c8bf4a85124e2b
Oct 30 21:42:04.872634 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:04.872600    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7b7bb4bc77-vqwsv" event={"ID":"b87c93be-5f5a-45ba-b820-ebd97c1a1c03","Type":"ContainerStarted","Data":"c6e984d8bf03dbd0ac0f943b5f78d77c1cd7aac76eb8570816c8bf4a85124e2b"}
Oct 30 21:42:05.024931 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:05.024842    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:05.540887 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:05.540846    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:05.566437 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:42:05.566398    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:42:07.321807 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:07.321770    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-pfgpx"]
Oct 30 21:42:07.326177 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:07.326152    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-pfgpx"
Oct 30 21:42:07.422892 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:07.422855    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-87p4g\""
Oct 30 21:42:07.425458 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:07.425427    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-pfgpx"]
Oct 30 21:42:07.465928 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:07.465888    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvj6k\" (UniqueName: \"kubernetes.io/projected/5754b4e2-2ccc-43ee-b1d2-e065d15345ca-kube-api-access-fvj6k\") pod \"limitador-operator-controller-manager-5778bb4449-pfgpx\" (UID: \"5754b4e2-2ccc-43ee-b1d2-e065d15345ca\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-pfgpx"
Oct 30 21:42:07.567407 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:07.567361    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvj6k\" (UniqueName: \"kubernetes.io/projected/5754b4e2-2ccc-43ee-b1d2-e065d15345ca-kube-api-access-fvj6k\") pod \"limitador-operator-controller-manager-5778bb4449-pfgpx\" (UID: \"5754b4e2-2ccc-43ee-b1d2-e065d15345ca\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-pfgpx"
Oct 30 21:42:07.640169 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:07.640129    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvj6k\" (UniqueName: \"kubernetes.io/projected/5754b4e2-2ccc-43ee-b1d2-e065d15345ca-kube-api-access-fvj6k\") pod \"limitador-operator-controller-manager-5778bb4449-pfgpx\" (UID: \"5754b4e2-2ccc-43ee-b1d2-e065d15345ca\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-pfgpx"
Oct 30 21:42:07.641763 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:07.641736    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-pfgpx"
Oct 30 21:42:08.032821 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:08.032784    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-pfgpx"]
Oct 30 21:42:08.033357 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:42:08.033326    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5754b4e2_2ccc_43ee_b1d2_e065d15345ca.slice/crio-af2149ab4c452eff6509ee44889ab448b207adee76fd09592f35c882d5022a2d WatchSource:0}: Error finding container af2149ab4c452eff6509ee44889ab448b207adee76fd09592f35c882d5022a2d: Status 404 returned error can't find the container with id af2149ab4c452eff6509ee44889ab448b207adee76fd09592f35c882d5022a2d
Oct 30 21:42:08.277879 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:08.277842    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:08.429131 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:08.429096    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:08.788058 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:08.788014    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:08.898413 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:08.898376    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7b7bb4bc77-vqwsv" event={"ID":"b87c93be-5f5a-45ba-b820-ebd97c1a1c03","Type":"ContainerStarted","Data":"933d2993e61c73746362397ee0b15300f8c81ee67e0513fc133a044e5e0c7923"}
Oct 30 21:42:08.898628 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:08.898439    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-7b7bb4bc77-vqwsv"
Oct 30 21:42:08.899790 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:08.899752    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-pfgpx" event={"ID":"5754b4e2-2ccc-43ee-b1d2-e065d15345ca","Type":"ContainerStarted","Data":"af2149ab4c452eff6509ee44889ab448b207adee76fd09592f35c882d5022a2d"}
Oct 30 21:42:09.009948 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:09.009889    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-7b7bb4bc77-vqwsv" podStartSLOduration=2.363674211 podStartE2EDuration="6.009874585s" totalImagesPullingTime="3.646200374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:42:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:42:09.009533032 +0000 UTC m=+271.187021647" watchObservedRunningTime="2026-10-30 21:42:09.009874585 +0000 UTC m=+271.187363202"
Oct 30 21:42:10.805715 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:10.805688    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:10.916336 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:10.916301    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-pfgpx" event={"ID":"5754b4e2-2ccc-43ee-b1d2-e065d15345ca","Type":"ContainerStarted","Data":"6c29943e58d38ee6705c7a6bbe21ad0cd63c1acd861ae1b1ed58d62bdc0655b0"}
Oct 30 21:42:10.916498 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:10.916463    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-pfgpx"
Oct 30 21:42:11.032044 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:11.031841    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-pfgpx" podStartSLOduration=1.307133313 podStartE2EDuration="4.031826575s" totalImagesPullingTime="2.724693262s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:42:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:42:11.029517754 +0000 UTC m=+273.207006370" watchObservedRunningTime="2026-10-30 21:42:11.031826575 +0000 UTC m=+273.209315193"
Oct 30 21:42:11.425534 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:11.425500    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:11.460319 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:11.460279    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-p8zgz"]
Oct 30 21:42:11.462972 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:11.462949    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-p8zgz"
Oct 30 21:42:11.516416 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:11.516383    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-t5qmp\""
Oct 30 21:42:11.518334 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:11.518306    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Oct 30 21:42:11.562723 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:11.562678    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-p8zgz"]
Oct 30 21:42:11.598948 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:11.598909    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gn7j\" (UniqueName: \"kubernetes.io/projected/9c392070-49cb-4c08-8230-19737bd80de4-kube-api-access-9gn7j\") pod \"dns-operator-controller-manager-8557cbdb84-p8zgz\" (UID: \"9c392070-49cb-4c08-8230-19737bd80de4\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-p8zgz"
Oct 30 21:42:11.609818 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:11.609779    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:11.700131 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:11.700022    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gn7j\" (UniqueName: \"kubernetes.io/projected/9c392070-49cb-4c08-8230-19737bd80de4-kube-api-access-9gn7j\") pod \"dns-operator-controller-manager-8557cbdb84-p8zgz\" (UID: \"9c392070-49cb-4c08-8230-19737bd80de4\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-p8zgz"
Oct 30 21:42:11.762205 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:11.762170    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gn7j\" (UniqueName: \"kubernetes.io/projected/9c392070-49cb-4c08-8230-19737bd80de4-kube-api-access-9gn7j\") pod \"dns-operator-controller-manager-8557cbdb84-p8zgz\" (UID: \"9c392070-49cb-4c08-8230-19737bd80de4\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-p8zgz"
Oct 30 21:42:11.775003 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:11.774964    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-p8zgz"
Oct 30 21:42:12.144733 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:12.144699    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-p8zgz"]
Oct 30 21:42:12.146304 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:42:12.146275    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c392070_49cb_4c08_8230_19737bd80de4.slice/crio-dc02b78bc451de4a1abb5a9278fde5386f0e29ac8de0b4cd324b8b260016a6ae WatchSource:0}: Error finding container dc02b78bc451de4a1abb5a9278fde5386f0e29ac8de0b4cd324b8b260016a6ae: Status 404 returned error can't find the container with id dc02b78bc451de4a1abb5a9278fde5386f0e29ac8de0b4cd324b8b260016a6ae
Oct 30 21:42:12.373586 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:12.373544    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:12.728805 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:12.728759    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:12.839323 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:12.839285    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-84bc87c977-7dw9z"]
Oct 30 21:42:12.843306 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:12.843270    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-84bc87c977-7dw9z"
Oct 30 21:42:12.902974 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:12.902930    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Oct 30 21:42:12.902974 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:12.902932    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Oct 30 21:42:12.903262 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:12.903104    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Oct 30 21:42:12.903342 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:12.903287    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Oct 30 21:42:12.903466 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:12.903448    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-nkddb\""
Oct 30 21:42:12.903527 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:12.903482    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Oct 30 21:42:12.928167 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:12.928129    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-p8zgz" event={"ID":"9c392070-49cb-4c08-8230-19737bd80de4","Type":"ContainerStarted","Data":"dc02b78bc451de4a1abb5a9278fde5386f0e29ac8de0b4cd324b8b260016a6ae"}
Oct 30 21:42:13.013410 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:13.013312    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cfd733d0-9bcf-452c-94bc-9259bb0278c9-tmp\") pod \"opendatahub-operator-controller-manager-84bc87c977-7dw9z\" (UID: \"cfd733d0-9bcf-452c-94bc-9259bb0278c9\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-84bc87c977-7dw9z"
Oct 30 21:42:13.013410 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:13.013368    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cfd733d0-9bcf-452c-94bc-9259bb0278c9-webhook-cert\") pod \"opendatahub-operator-controller-manager-84bc87c977-7dw9z\" (UID: \"cfd733d0-9bcf-452c-94bc-9259bb0278c9\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-84bc87c977-7dw9z"
Oct 30 21:42:13.013662 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:13.013467    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2tpp\" (UniqueName: \"kubernetes.io/projected/cfd733d0-9bcf-452c-94bc-9259bb0278c9-kube-api-access-x2tpp\") pod \"opendatahub-operator-controller-manager-84bc87c977-7dw9z\" (UID: \"cfd733d0-9bcf-452c-94bc-9259bb0278c9\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-84bc87c977-7dw9z"
Oct 30 21:42:13.013662 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:13.013502    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cfd733d0-9bcf-452c-94bc-9259bb0278c9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84bc87c977-7dw9z\" (UID: \"cfd733d0-9bcf-452c-94bc-9259bb0278c9\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-84bc87c977-7dw9z"
Oct 30 21:42:13.013662 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:13.013590    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cfd733d0-9bcf-452c-94bc-9259bb0278c9-manifests\") pod \"opendatahub-operator-controller-manager-84bc87c977-7dw9z\" (UID: \"cfd733d0-9bcf-452c-94bc-9259bb0278c9\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-84bc87c977-7dw9z"
Oct 30 21:42:13.013662 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:13.013651    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/cfd733d0-9bcf-452c-94bc-9259bb0278c9-metrics-cert\") pod \"opendatahub-operator-controller-manager-84bc87c977-7dw9z\" (UID: \"cfd733d0-9bcf-452c-94bc-9259bb0278c9\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-84bc87c977-7dw9z"
Oct 30 21:42:13.056255 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:13.056197    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-84bc87c977-7dw9z"]
Oct 30 21:42:13.114975 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:13.114934    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2tpp\" (UniqueName: \"kubernetes.io/projected/cfd733d0-9bcf-452c-94bc-9259bb0278c9-kube-api-access-x2tpp\") pod \"opendatahub-operator-controller-manager-84bc87c977-7dw9z\" (UID: \"cfd733d0-9bcf-452c-94bc-9259bb0278c9\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-84bc87c977-7dw9z"
Oct 30 21:42:13.115171 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:13.114981    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cfd733d0-9bcf-452c-94bc-9259bb0278c9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84bc87c977-7dw9z\" (UID: \"cfd733d0-9bcf-452c-94bc-9259bb0278c9\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-84bc87c977-7dw9z"
Oct 30 21:42:13.115171 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:13.115013    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cfd733d0-9bcf-452c-94bc-9259bb0278c9-manifests\") pod \"opendatahub-operator-controller-manager-84bc87c977-7dw9z\" (UID: \"cfd733d0-9bcf-452c-94bc-9259bb0278c9\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-84bc87c977-7dw9z"
Oct 30 21:42:13.115171 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:13.115039    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/cfd733d0-9bcf-452c-94bc-9259bb0278c9-metrics-cert\") pod \"opendatahub-operator-controller-manager-84bc87c977-7dw9z\" (UID: \"cfd733d0-9bcf-452c-94bc-9259bb0278c9\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-84bc87c977-7dw9z"
Oct 30 21:42:13.115433 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:13.115173    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cfd733d0-9bcf-452c-94bc-9259bb0278c9-tmp\") pod \"opendatahub-operator-controller-manager-84bc87c977-7dw9z\" (UID: \"cfd733d0-9bcf-452c-94bc-9259bb0278c9\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-84bc87c977-7dw9z"
Oct 30 21:42:13.115433 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:13.115264    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cfd733d0-9bcf-452c-94bc-9259bb0278c9-webhook-cert\") pod \"opendatahub-operator-controller-manager-84bc87c977-7dw9z\" (UID: \"cfd733d0-9bcf-452c-94bc-9259bb0278c9\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-84bc87c977-7dw9z"
Oct 30 21:42:13.115547 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:13.115459    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cfd733d0-9bcf-452c-94bc-9259bb0278c9-manifests\") pod \"opendatahub-operator-controller-manager-84bc87c977-7dw9z\" (UID: \"cfd733d0-9bcf-452c-94bc-9259bb0278c9\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-84bc87c977-7dw9z"
Oct 30 21:42:13.117608 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:13.117577    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cfd733d0-9bcf-452c-94bc-9259bb0278c9-tmp\") pod \"opendatahub-operator-controller-manager-84bc87c977-7dw9z\" (UID: \"cfd733d0-9bcf-452c-94bc-9259bb0278c9\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-84bc87c977-7dw9z"
Oct 30 21:42:13.117738 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:13.117703    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/cfd733d0-9bcf-452c-94bc-9259bb0278c9-metrics-cert\") pod \"opendatahub-operator-controller-manager-84bc87c977-7dw9z\" (UID: \"cfd733d0-9bcf-452c-94bc-9259bb0278c9\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-84bc87c977-7dw9z"
Oct 30 21:42:13.117878 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:13.117858    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cfd733d0-9bcf-452c-94bc-9259bb0278c9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84bc87c977-7dw9z\" (UID: \"cfd733d0-9bcf-452c-94bc-9259bb0278c9\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-84bc87c977-7dw9z"
Oct 30 21:42:13.117878 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:13.117868    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cfd733d0-9bcf-452c-94bc-9259bb0278c9-webhook-cert\") pod \"opendatahub-operator-controller-manager-84bc87c977-7dw9z\" (UID: \"cfd733d0-9bcf-452c-94bc-9259bb0278c9\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-84bc87c977-7dw9z"
Oct 30 21:42:13.147264 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:13.147217    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:13.173681 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:13.173645    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2tpp\" (UniqueName: \"kubernetes.io/projected/cfd733d0-9bcf-452c-94bc-9259bb0278c9-kube-api-access-x2tpp\") pod \"opendatahub-operator-controller-manager-84bc87c977-7dw9z\" (UID: \"cfd733d0-9bcf-452c-94bc-9259bb0278c9\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-84bc87c977-7dw9z"
Oct 30 21:42:13.318625 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:13.318538    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:13.459978 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:13.459936    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-84bc87c977-7dw9z"
Oct 30 21:42:13.492479 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:13.492441    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:13.944375 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:13.944341    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-84bc87c977-7dw9z"]
Oct 30 21:42:13.994382 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:42:13.994344    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfd733d0_9bcf_452c_94bc_9259bb0278c9.slice/crio-e042a909b1be84e0eb3bc47bfba38401eb2713e73c1d146f3e756aad45a567a8 WatchSource:0}: Error finding container e042a909b1be84e0eb3bc47bfba38401eb2713e73c1d146f3e756aad45a567a8: Status 404 returned error can't find the container with id e042a909b1be84e0eb3bc47bfba38401eb2713e73c1d146f3e756aad45a567a8
Oct 30 21:42:14.009420 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:14.009389    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:14.941222 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:14.941178    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-84bc87c977-7dw9z" event={"ID":"cfd733d0-9bcf-452c-94bc-9259bb0278c9","Type":"ContainerStarted","Data":"e042a909b1be84e0eb3bc47bfba38401eb2713e73c1d146f3e756aad45a567a8"}
Oct 30 21:42:15.138391 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:15.138355    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:15.307684 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:15.307646    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:15.951629 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:15.951582    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-p8zgz" event={"ID":"9c392070-49cb-4c08-8230-19737bd80de4","Type":"ContainerStarted","Data":"f8e4d960c84c53af95560dce64d9ceede55c8219c8ab063164baca6b5ec996db"}
Oct 30 21:42:15.952142 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:15.951698    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-p8zgz"
Oct 30 21:42:16.104351 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:16.104289    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-p8zgz" podStartSLOduration=2.077501431 podStartE2EDuration="5.104273347s" totalImagesPullingTime="3.026771916s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:42:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:42:16.101139409 +0000 UTC m=+278.278628025" watchObservedRunningTime="2026-10-30 21:42:16.104273347 +0000 UTC m=+278.281761962"
Oct 30 21:42:17.965567 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:17.965532    4871 generic.go:356] "Generic (PLEG): container finished" podID="cfd733d0-9bcf-452c-94bc-9259bb0278c9" containerID="dcd741ef0b1011249f079907221d5050892b73c5a37878f9956d34a774b6c23a" exitCode=0
Oct 30 21:42:17.965983 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:17.965577    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-84bc87c977-7dw9z" event={"ID":"cfd733d0-9bcf-452c-94bc-9259bb0278c9","Type":"ContainerDied","Data":"dcd741ef0b1011249f079907221d5050892b73c5a37878f9956d34a774b6c23a"}
Oct 30 21:42:19.097179 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:19.097140    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-r5c8j"]
Oct 30 21:42:19.100702 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:19.100681    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-r5c8j"
Oct 30 21:42:19.158255 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:19.158208    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-s2r8s\""
Oct 30 21:42:19.162313 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:19.162281    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-r5c8j"]
Oct 30 21:42:19.276582 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:19.276483    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69d6p\" (UniqueName: \"kubernetes.io/projected/4f069634-b231-4fd0-847e-52cb47ef0163-kube-api-access-69d6p\") pod \"kuadrant-operator-controller-manager-678d87766f-r5c8j\" (UID: \"4f069634-b231-4fd0-847e-52cb47ef0163\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-r5c8j"
Oct 30 21:42:19.276764 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:19.276638    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f069634-b231-4fd0-847e-52cb47ef0163-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-r5c8j\" (UID: \"4f069634-b231-4fd0-847e-52cb47ef0163\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-r5c8j"
Oct 30 21:42:19.377401 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:19.377347    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f069634-b231-4fd0-847e-52cb47ef0163-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-r5c8j\" (UID: \"4f069634-b231-4fd0-847e-52cb47ef0163\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-r5c8j"
Oct 30 21:42:19.377618 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:19.377431    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69d6p\" (UniqueName: \"kubernetes.io/projected/4f069634-b231-4fd0-847e-52cb47ef0163-kube-api-access-69d6p\") pod \"kuadrant-operator-controller-manager-678d87766f-r5c8j\" (UID: \"4f069634-b231-4fd0-847e-52cb47ef0163\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-r5c8j"
Oct 30 21:42:19.377819 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:19.377794    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f069634-b231-4fd0-847e-52cb47ef0163-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-r5c8j\" (UID: \"4f069634-b231-4fd0-847e-52cb47ef0163\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-r5c8j"
Oct 30 21:42:19.457698 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:19.457661    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-69d6p\" (UniqueName: \"kubernetes.io/projected/4f069634-b231-4fd0-847e-52cb47ef0163-kube-api-access-69d6p\") pod \"kuadrant-operator-controller-manager-678d87766f-r5c8j\" (UID: \"4f069634-b231-4fd0-847e-52cb47ef0163\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-r5c8j"
Oct 30 21:42:19.712918 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:19.712880    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-r5c8j"
Oct 30 21:42:19.911220 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:19.911188    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-7b7bb4bc77-vqwsv"
Oct 30 21:42:19.979742 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:19.979652    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-84bc87c977-7dw9z" event={"ID":"cfd733d0-9bcf-452c-94bc-9259bb0278c9","Type":"ContainerStarted","Data":"1ba771bc2e7cfd5a9cb25a515170bf5c280132905a69d1b21f2e4ad117b4a545"}
Oct 30 21:42:19.979892 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:19.979781    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-84bc87c977-7dw9z"
Oct 30 21:42:20.129730 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:20.129688    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:20.202171 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:20.202145    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-r5c8j"]
Oct 30 21:42:20.204320 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:42:20.204277    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f069634_b231_4fd0_847e_52cb47ef0163.slice/crio-e016820c04dcc0f3a917bc8787d96347b877cbd1d7a179ac985313698929afc5 WatchSource:0}: Error finding container e016820c04dcc0f3a917bc8787d96347b877cbd1d7a179ac985313698929afc5: Status 404 returned error can't find the container with id e016820c04dcc0f3a917bc8787d96347b877cbd1d7a179ac985313698929afc5
Oct 30 21:42:20.249608 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:20.249475    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-84bc87c977-7dw9z" podStartSLOduration=3.444014813 podStartE2EDuration="8.249455728s" totalImagesPullingTime="4.377940056s" totalInitContainerRuntime="427.500859ms" isStatefulPod=false podCreationTimestamp="2026-10-30 21:42:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:42:20.221231087 +0000 UTC m=+282.398719697" watchObservedRunningTime="2026-10-30 21:42:20.249455728 +0000 UTC m=+282.426944348"
Oct 30 21:42:20.398448 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:20.398409    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:20.572007 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:20.571919    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:20.827589 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:20.827501    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:20.986214 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:20.986177    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-r5c8j" event={"ID":"4f069634-b231-4fd0-847e-52cb47ef0163","Type":"ContainerStarted","Data":"e016820c04dcc0f3a917bc8787d96347b877cbd1d7a179ac985313698929afc5"}
Oct 30 21:42:21.022715 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:21.022676    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:21.924596 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:21.924564    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-pfgpx"
Oct 30 21:42:25.559770 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:25.559734    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:25.710278 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:25.710200    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:26.016947 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:26.016908    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-r5c8j" event={"ID":"4f069634-b231-4fd0-847e-52cb47ef0163","Type":"ContainerStarted","Data":"0ceafaaf94068190c4d1656463de440f70dd7913d0c0bb267466517724a8b136"}
Oct 30 21:42:26.017134 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:26.016979    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-r5c8j"
Oct 30 21:42:26.222969 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:26.222884    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-r5c8j" podStartSLOduration=1.7568727530000001 podStartE2EDuration="7.222863615s" totalImagesPullingTime="5.465990862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:42:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:42:26.222785074 +0000 UTC m=+288.400273691" watchObservedRunningTime="2026-10-30 21:42:26.222863615 +0000 UTC m=+288.400352226"
Oct 30 21:42:26.960274 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:26.960223    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-p8zgz"
Oct 30 21:42:28.985530 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:28.985479    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:29.135123 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:29.135086    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:29.314666 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:29.314565    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:29.844616 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:29.844574    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:29.992098 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:29.992057    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:30.989115 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:30.989084    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-84bc87c977-7dw9z"
Oct 30 21:42:33.842445 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:33.842411    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:33.999814 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:33.999764    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:35.567582 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:42:35.567546    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:42:37.025070 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:37.025041    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-r5c8j"
Oct 30 21:42:37.983198 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:37.983169    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx_8d401654-8a4c-4a79-b2d3-612765312421/extract/0.log"
Oct 30 21:42:37.983391 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:37.983169    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx_8d401654-8a4c-4a79-b2d3-612765312421/extract/0.log"
Oct 30 21:42:37.995841 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:37.995817    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/2.log"
Oct 30 21:42:37.996003 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:37.995822    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/2.log"
Oct 30 21:42:38.003451 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:38.003422    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzntd_2360fddc-52a5-42a5-82ca-a05aff84d217/ovn-acl-logging/0.log"
Oct 30 21:42:38.003658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:38.003642    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzntd_2360fddc-52a5-42a5-82ca-a05aff84d217/ovn-acl-logging/0.log"
Oct 30 21:42:40.877764 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:40.877720    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:41.036225 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:41.036180    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:44.456206 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:44.456163    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:44.616394 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:44.616356    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:47.455525 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:47.455488    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:47.612800 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:47.612767    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:51.588933 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:51.588846    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:51.760167 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:51.760135    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:52.923490 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:52.923451    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:53.146151 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:53.146116    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:53.296150 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:53.296063    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:53.933117 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:42:53.933083    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:04.460043 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:04.460008    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:04.764893 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:04.764810    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:04.928180 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:04.928145    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:05.568953 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:43:05.568914    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:43:06.535861 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:06.535822    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:06.742338 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:06.742303    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:06.902604 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:06.902569    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:07.108025 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:07.107980    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-575956cc9c6mmf7"]
Oct 30 21:43:07.114565 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:07.114535    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-575956cc9c6mmf7"
Oct 30 21:43:07.168823 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:07.168729    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-6jdt7\""
Oct 30 21:43:07.196956 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:07.196917    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:07.217521 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:07.217474    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-575956cc9c6mmf7"]
Oct 30 21:43:07.295942 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:07.295902    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3419e5d7-f2ed-43e1-89e0-baccdb485f1b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-575956cc9c6mmf7\" (UID: \"3419e5d7-f2ed-43e1-89e0-baccdb485f1b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-575956cc9c6mmf7"
Oct 30 21:43:07.296138 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:07.295948    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3419e5d7-f2ed-43e1-89e0-baccdb485f1b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-575956cc9c6mmf7\" (UID: \"3419e5d7-f2ed-43e1-89e0-baccdb485f1b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-575956cc9c6mmf7"
Oct 30 21:43:07.296138 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:07.296032    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3419e5d7-f2ed-43e1-89e0-baccdb485f1b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-575956cc9c6mmf7\" (UID: \"3419e5d7-f2ed-43e1-89e0-baccdb485f1b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-575956cc9c6mmf7"
Oct 30 21:43:07.296138 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:07.296075    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3419e5d7-f2ed-43e1-89e0-baccdb485f1b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-575956cc9c6mmf7\" (UID: \"3419e5d7-f2ed-43e1-89e0-baccdb485f1b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-575956cc9c6mmf7"
Oct 30 21:43:07.296138 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:07.296095    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pzhd\" (UniqueName: \"kubernetes.io/projected/3419e5d7-f2ed-43e1-89e0-baccdb485f1b-kube-api-access-9pzhd\") pod \"data-science-gateway-data-science-gateway-class-575956cc9c6mmf7\" (UID: \"3419e5d7-f2ed-43e1-89e0-baccdb485f1b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-575956cc9c6mmf7"
Oct 30 21:43:07.296396 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:07.296189    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3419e5d7-f2ed-43e1-89e0-baccdb485f1b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-575956cc9c6mmf7\" (UID: \"3419e5d7-f2ed-43e1-89e0-baccdb485f1b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-575956cc9c6mmf7"
Oct 30 21:43:07.296396 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:07.296324    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3419e5d7-f2ed-43e1-89e0-baccdb485f1b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-575956cc9c6mmf7\" (UID: \"3419e5d7-f2ed-43e1-89e0-baccdb485f1b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-575956cc9c6mmf7"
Oct 30 21:43:07.296396 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:07.296374    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3419e5d7-f2ed-43e1-89e0-baccdb485f1b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-575956cc9c6mmf7\" (UID: \"3419e5d7-f2ed-43e1-89e0-baccdb485f1b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-575956cc9c6mmf7"
Oct 30 21:43:07.296498 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:07.296425    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3419e5d7-f2ed-43e1-89e0-baccdb485f1b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-575956cc9c6mmf7\" (UID: \"3419e5d7-f2ed-43e1-89e0-baccdb485f1b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-575956cc9c6mmf7"
Oct 30 21:43:07.369769 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:07.369738    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:07.397799 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:07.397760    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3419e5d7-f2ed-43e1-89e0-baccdb485f1b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-575956cc9c6mmf7\" (UID: \"3419e5d7-f2ed-43e1-89e0-baccdb485f1b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-575956cc9c6mmf7"
Oct 30 21:43:07.398003 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:07.397805    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3419e5d7-f2ed-43e1-89e0-baccdb485f1b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-575956cc9c6mmf7\" (UID: \"3419e5d7-f2ed-43e1-89e0-baccdb485f1b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-575956cc9c6mmf7"
Oct 30 21:43:07.398003 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:07.397968    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3419e5d7-f2ed-43e1-89e0-baccdb485f1b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-575956cc9c6mmf7\" (UID: \"3419e5d7-f2ed-43e1-89e0-baccdb485f1b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-575956cc9c6mmf7"
Oct 30 21:43:07.398127 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:07.398033    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3419e5d7-f2ed-43e1-89e0-baccdb485f1b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-575956cc9c6mmf7\" (UID: \"3419e5d7-f2ed-43e1-89e0-baccdb485f1b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-575956cc9c6mmf7"
Oct 30 21:43:07.398127 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:07.398065    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3419e5d7-f2ed-43e1-89e0-baccdb485f1b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-575956cc9c6mmf7\" (UID: \"3419e5d7-f2ed-43e1-89e0-baccdb485f1b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-575956cc9c6mmf7"
Oct 30 21:43:07.398127 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:07.398089    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3419e5d7-f2ed-43e1-89e0-baccdb485f1b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-575956cc9c6mmf7\" (UID: \"3419e5d7-f2ed-43e1-89e0-baccdb485f1b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-575956cc9c6mmf7"
Oct 30 21:43:07.398127 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:07.398123    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3419e5d7-f2ed-43e1-89e0-baccdb485f1b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-575956cc9c6mmf7\" (UID: \"3419e5d7-f2ed-43e1-89e0-baccdb485f1b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-575956cc9c6mmf7"
Oct 30 21:43:07.398360 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:07.398142    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pzhd\" (UniqueName: \"kubernetes.io/projected/3419e5d7-f2ed-43e1-89e0-baccdb485f1b-kube-api-access-9pzhd\") pod \"data-science-gateway-data-science-gateway-class-575956cc9c6mmf7\" (UID: \"3419e5d7-f2ed-43e1-89e0-baccdb485f1b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-575956cc9c6mmf7"
Oct 30 21:43:07.398360 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:07.398182    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3419e5d7-f2ed-43e1-89e0-baccdb485f1b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-575956cc9c6mmf7\" (UID: \"3419e5d7-f2ed-43e1-89e0-baccdb485f1b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-575956cc9c6mmf7"
Oct 30 21:43:07.398476 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:07.398448    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3419e5d7-f2ed-43e1-89e0-baccdb485f1b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-575956cc9c6mmf7\" (UID: \"3419e5d7-f2ed-43e1-89e0-baccdb485f1b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-575956cc9c6mmf7"
Oct 30 21:43:07.398534 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:07.398491    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3419e5d7-f2ed-43e1-89e0-baccdb485f1b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-575956cc9c6mmf7\" (UID: \"3419e5d7-f2ed-43e1-89e0-baccdb485f1b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-575956cc9c6mmf7"
Oct 30 21:43:07.398534 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:07.398518    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3419e5d7-f2ed-43e1-89e0-baccdb485f1b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-575956cc9c6mmf7\" (UID: \"3419e5d7-f2ed-43e1-89e0-baccdb485f1b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-575956cc9c6mmf7"
Oct 30 21:43:07.398734 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:07.398716    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3419e5d7-f2ed-43e1-89e0-baccdb485f1b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-575956cc9c6mmf7\" (UID: \"3419e5d7-f2ed-43e1-89e0-baccdb485f1b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-575956cc9c6mmf7"
Oct 30 21:43:07.398909 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:07.398880    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3419e5d7-f2ed-43e1-89e0-baccdb485f1b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-575956cc9c6mmf7\" (UID: \"3419e5d7-f2ed-43e1-89e0-baccdb485f1b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-575956cc9c6mmf7"
Oct 30 21:43:07.400206 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:07.400184    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3419e5d7-f2ed-43e1-89e0-baccdb485f1b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-575956cc9c6mmf7\" (UID: \"3419e5d7-f2ed-43e1-89e0-baccdb485f1b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-575956cc9c6mmf7"
Oct 30 21:43:07.400515 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:07.400494    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3419e5d7-f2ed-43e1-89e0-baccdb485f1b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-575956cc9c6mmf7\" (UID: \"3419e5d7-f2ed-43e1-89e0-baccdb485f1b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-575956cc9c6mmf7"
Oct 30 21:43:07.456359 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:07.456271    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3419e5d7-f2ed-43e1-89e0-baccdb485f1b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-575956cc9c6mmf7\" (UID: \"3419e5d7-f2ed-43e1-89e0-baccdb485f1b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-575956cc9c6mmf7"
Oct 30 21:43:07.456614 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:07.456589    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pzhd\" (UniqueName: \"kubernetes.io/projected/3419e5d7-f2ed-43e1-89e0-baccdb485f1b-kube-api-access-9pzhd\") pod \"data-science-gateway-data-science-gateway-class-575956cc9c6mmf7\" (UID: \"3419e5d7-f2ed-43e1-89e0-baccdb485f1b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-575956cc9c6mmf7"
Oct 30 21:43:07.521429 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:07.521395    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:07.726775 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:07.726676    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-575956cc9c6mmf7"
Oct 30 21:43:08.155185 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:08.155156    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-575956cc9c6mmf7"]
Oct 30 21:43:08.156847 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:43:08.156811    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3419e5d7_f2ed_43e1_89e0_baccdb485f1b.slice/crio-032bccbba6aca6f23b5f2451ab936128afa09e9efbcd36fc29a95d3709968700 WatchSource:0}: Error finding container 032bccbba6aca6f23b5f2451ab936128afa09e9efbcd36fc29a95d3709968700: Status 404 returned error can't find the container with id 032bccbba6aca6f23b5f2451ab936128afa09e9efbcd36fc29a95d3709968700
Oct 30 21:43:08.159486 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:08.159445    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:43:08.159613 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:08.159531    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:43:08.159613 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:08.159573    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:43:08.248654 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:08.248612    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-575956cc9c6mmf7" event={"ID":"3419e5d7-f2ed-43e1-89e0-baccdb485f1b","Type":"ContainerStarted","Data":"a308b2525ac75962df05935cf5139cb1006d8d8bed8369702080ade1cad27b55"}
Oct 30 21:43:08.248823 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:08.248687    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-575956cc9c6mmf7" event={"ID":"3419e5d7-f2ed-43e1-89e0-baccdb485f1b","Type":"ContainerStarted","Data":"032bccbba6aca6f23b5f2451ab936128afa09e9efbcd36fc29a95d3709968700"}
Oct 30 21:43:08.366177 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:08.366109    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-575956cc9c6mmf7" podStartSLOduration=1.3660906800000001 podStartE2EDuration="1.36609068s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:43:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:43:08.363450336 +0000 UTC m=+330.540938966" watchObservedRunningTime="2026-10-30 21:43:08.36609068 +0000 UTC m=+330.543579300"
Oct 30 21:43:08.727708 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:08.727666    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-575956cc9c6mmf7"
Oct 30 21:43:08.784383 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:08.784344    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:09.140715 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:09.140678    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:09.291949 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:09.291912    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:09.732152 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:09.732121    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-575956cc9c6mmf7"
Oct 30 21:43:10.258750 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:10.258716    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-575956cc9c6mmf7"
Oct 30 21:43:10.259777 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:10.259754    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-575956cc9c6mmf7"
Oct 30 21:43:10.339881 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:10.339849    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-57f7f8659f-969ld"]
Oct 30 21:43:10.342355 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:10.342335    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-969ld"
Oct 30 21:43:10.394000 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:10.393966    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-kd7kz\""
Oct 30 21:43:10.396360 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:10.396333    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-969ld"]
Oct 30 21:43:10.528385 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:10.528303    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7b94t\" (UniqueName: \"kubernetes.io/projected/10913847-1d45-4524-968b-9dc7e37d1f42-kube-api-access-7b94t\") pod \"authorino-57f7f8659f-969ld\" (UID: \"10913847-1d45-4524-968b-9dc7e37d1f42\") " pod="kuadrant-system/authorino-57f7f8659f-969ld"
Oct 30 21:43:10.590532 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:10.590496    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:10.629729 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:10.629692    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7b94t\" (UniqueName: \"kubernetes.io/projected/10913847-1d45-4524-968b-9dc7e37d1f42-kube-api-access-7b94t\") pod \"authorino-57f7f8659f-969ld\" (UID: \"10913847-1d45-4524-968b-9dc7e37d1f42\") " pod="kuadrant-system/authorino-57f7f8659f-969ld"
Oct 30 21:43:10.688306 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:10.688270    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7b94t\" (UniqueName: \"kubernetes.io/projected/10913847-1d45-4524-968b-9dc7e37d1f42-kube-api-access-7b94t\") pod \"authorino-57f7f8659f-969ld\" (UID: \"10913847-1d45-4524-968b-9dc7e37d1f42\") " pod="kuadrant-system/authorino-57f7f8659f-969ld"
Oct 30 21:43:10.761309 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:10.761270    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:10.952925 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:10.952892    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-969ld"
Oct 30 21:43:11.317957 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:11.317927    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-969ld"]
Oct 30 21:43:11.319686 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:43:11.319653    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod10913847_1d45_4524_968b_9dc7e37d1f42.slice/crio-811f493783c5d3f470b2a881d2fc506de16ef89f6aa9757260789f534255662f WatchSource:0}: Error finding container 811f493783c5d3f470b2a881d2fc506de16ef89f6aa9757260789f534255662f: Status 404 returned error can't find the container with id 811f493783c5d3f470b2a881d2fc506de16ef89f6aa9757260789f534255662f
Oct 30 21:43:11.321107 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:11.321088    4871 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:43:12.270094 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:12.270052    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-969ld" event={"ID":"10913847-1d45-4524-968b-9dc7e37d1f42","Type":"ContainerStarted","Data":"811f493783c5d3f470b2a881d2fc506de16ef89f6aa9757260789f534255662f"}
Oct 30 21:43:14.032617 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:14.032575    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:15.288856 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:15.288816    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-969ld" event={"ID":"10913847-1d45-4524-968b-9dc7e37d1f42","Type":"ContainerStarted","Data":"30ad534c507d9a2ffa791925d33a970eb99c6943b9d31c2a5add282033e29b32"}
Oct 30 21:43:15.398059 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:15.398001    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-57f7f8659f-969ld" podStartSLOduration=2.301956217 podStartE2EDuration="5.397986881s" totalImagesPullingTime="3.096030664s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:43:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:43:15.396707398 +0000 UTC m=+337.574196014" watchObservedRunningTime="2026-10-30 21:43:15.397986881 +0000 UTC m=+337.575475498"
Oct 30 21:43:16.538205 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:16.538169    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-969ld"]
Oct 30 21:43:17.299659 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:17.299620    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-57f7f8659f-969ld" podUID="10913847-1d45-4524-968b-9dc7e37d1f42" containerName="authorino" containerID="cri-o://30ad534c507d9a2ffa791925d33a970eb99c6943b9d31c2a5add282033e29b32" gracePeriod=30
Oct 30 21:43:17.552597 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:17.552532    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-969ld"
Oct 30 21:43:17.590352 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:17.590309    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/10913847-1d45-4524-968b-9dc7e37d1f42-kube-api-access-7b94t\" (UniqueName: \"kubernetes.io/projected/10913847-1d45-4524-968b-9dc7e37d1f42-kube-api-access-7b94t\") pod \"10913847-1d45-4524-968b-9dc7e37d1f42\" (UID: \"10913847-1d45-4524-968b-9dc7e37d1f42\") "
Oct 30 21:43:17.592637 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:17.592603    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10913847-1d45-4524-968b-9dc7e37d1f42-kube-api-access-7b94t" pod "10913847-1d45-4524-968b-9dc7e37d1f42" (UID: "10913847-1d45-4524-968b-9dc7e37d1f42"). InnerVolumeSpecName "kube-api-access-7b94t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:43:17.691342 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:17.691299    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7b94t\" (UniqueName: \"kubernetes.io/projected/10913847-1d45-4524-968b-9dc7e37d1f42-kube-api-access-7b94t\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:43:17.741930 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:17.741892    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-qzpg5"]
Oct 30 21:43:17.742310 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:17.742294    4871 cpu_manager.go:414] "containerMap: removing container" podUID="10913847-1d45-4524-968b-9dc7e37d1f42" containerName="authorino"
Oct 30 21:43:17.742384 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:17.742323    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="10913847-1d45-4524-968b-9dc7e37d1f42" containerName="authorino"
Oct 30 21:43:17.742431 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:17.742414    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="10913847-1d45-4524-968b-9dc7e37d1f42" containerName="authorino"
Oct 30 21:43:17.745888 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:17.745866    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-qzpg5"
Oct 30 21:43:17.792745 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:17.792708    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnz28\" (UniqueName: \"kubernetes.io/projected/b7de093c-a605-4406-a940-e3e59abcb9a0-kube-api-access-xnz28\") pod \"kserve-module-controller-manager-7566cf5fc8-qzpg5\" (UID: \"b7de093c-a605-4406-a940-e3e59abcb9a0\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-qzpg5"
Oct 30 21:43:17.792933 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:17.792766    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b7de093c-a605-4406-a940-e3e59abcb9a0-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-qzpg5\" (UID: \"b7de093c-a605-4406-a940-e3e59abcb9a0\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-qzpg5"
Oct 30 21:43:17.792933 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:17.792791    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b7de093c-a605-4406-a940-e3e59abcb9a0-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-qzpg5\" (UID: \"b7de093c-a605-4406-a940-e3e59abcb9a0\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-qzpg5"
Oct 30 21:43:17.793024 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:17.792920    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b7de093c-a605-4406-a940-e3e59abcb9a0-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-qzpg5\" (UID: \"b7de093c-a605-4406-a940-e3e59abcb9a0\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-qzpg5"
Oct 30 21:43:17.795468 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:17.795446    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 30 21:43:17.795468 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:17.795470    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-module-controller-manager-dockercfg-wlkhg\""
Oct 30 21:43:17.795616 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:17.795487    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 30 21:43:17.862895 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:17.862857    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:17.890310 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:17.890276    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"odh-trusted-ca-bundle\""
Oct 30 21:43:17.892709 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:17.892687    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-qzpg5"]
Oct 30 21:43:17.893854 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:17.893831    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnz28\" (UniqueName: \"kubernetes.io/projected/b7de093c-a605-4406-a940-e3e59abcb9a0-kube-api-access-xnz28\") pod \"kserve-module-controller-manager-7566cf5fc8-qzpg5\" (UID: \"b7de093c-a605-4406-a940-e3e59abcb9a0\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-qzpg5"
Oct 30 21:43:17.893968 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:17.893896    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b7de093c-a605-4406-a940-e3e59abcb9a0-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-qzpg5\" (UID: \"b7de093c-a605-4406-a940-e3e59abcb9a0\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-qzpg5"
Oct 30 21:43:17.893968 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:17.893929    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b7de093c-a605-4406-a940-e3e59abcb9a0-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-qzpg5\" (UID: \"b7de093c-a605-4406-a940-e3e59abcb9a0\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-qzpg5"
Oct 30 21:43:17.893968 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:17.893962    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b7de093c-a605-4406-a940-e3e59abcb9a0-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-qzpg5\" (UID: \"b7de093c-a605-4406-a940-e3e59abcb9a0\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-qzpg5"
Oct 30 21:43:17.894381 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:17.894358    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b7de093c-a605-4406-a940-e3e59abcb9a0-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-qzpg5\" (UID: \"b7de093c-a605-4406-a940-e3e59abcb9a0\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-qzpg5"
Oct 30 21:43:17.894476 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:17.894397    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b7de093c-a605-4406-a940-e3e59abcb9a0-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-qzpg5\" (UID: \"b7de093c-a605-4406-a940-e3e59abcb9a0\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-qzpg5"
Oct 30 21:43:17.895050 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:17.895027    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b7de093c-a605-4406-a940-e3e59abcb9a0-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-qzpg5\" (UID: \"b7de093c-a605-4406-a940-e3e59abcb9a0\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-qzpg5"
Oct 30 21:43:17.952127 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:17.952096    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnz28\" (UniqueName: \"kubernetes.io/projected/b7de093c-a605-4406-a940-e3e59abcb9a0-kube-api-access-xnz28\") pod \"kserve-module-controller-manager-7566cf5fc8-qzpg5\" (UID: \"b7de093c-a605-4406-a940-e3e59abcb9a0\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-qzpg5"
Oct 30 21:43:18.058659 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:18.058623    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-qzpg5"
Oct 30 21:43:18.307618 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:18.307585    4871 generic.go:356] "Generic (PLEG): container finished" podID="10913847-1d45-4524-968b-9dc7e37d1f42" containerID="30ad534c507d9a2ffa791925d33a970eb99c6943b9d31c2a5add282033e29b32" exitCode=0
Oct 30 21:43:18.307809 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:18.307630    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-969ld"
Oct 30 21:43:18.307809 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:18.307649    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-969ld" event={"ID":"10913847-1d45-4524-968b-9dc7e37d1f42","Type":"ContainerDied","Data":"30ad534c507d9a2ffa791925d33a970eb99c6943b9d31c2a5add282033e29b32"}
Oct 30 21:43:18.307809 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:18.307690    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-969ld" event={"ID":"10913847-1d45-4524-968b-9dc7e37d1f42","Type":"ContainerDied","Data":"811f493783c5d3f470b2a881d2fc506de16ef89f6aa9757260789f534255662f"}
Oct 30 21:43:18.307809 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:18.307712    4871 scope.go:122] "RemoveContainer" containerID="30ad534c507d9a2ffa791925d33a970eb99c6943b9d31c2a5add282033e29b32"
Oct 30 21:43:18.323943 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:18.323917    4871 scope.go:122] "RemoveContainer" containerID="30ad534c507d9a2ffa791925d33a970eb99c6943b9d31c2a5add282033e29b32"
Oct 30 21:43:18.324294 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:43:18.324269    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30ad534c507d9a2ffa791925d33a970eb99c6943b9d31c2a5add282033e29b32\": container with ID starting with 30ad534c507d9a2ffa791925d33a970eb99c6943b9d31c2a5add282033e29b32 not found: ID does not exist" containerID="30ad534c507d9a2ffa791925d33a970eb99c6943b9d31c2a5add282033e29b32"
Oct 30 21:43:18.324354 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:18.324305    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30ad534c507d9a2ffa791925d33a970eb99c6943b9d31c2a5add282033e29b32"} err="failed to get container status \"30ad534c507d9a2ffa791925d33a970eb99c6943b9d31c2a5add282033e29b32\": rpc error: code = NotFound desc = could not find container \"30ad534c507d9a2ffa791925d33a970eb99c6943b9d31c2a5add282033e29b32\": container with ID starting with 30ad534c507d9a2ffa791925d33a970eb99c6943b9d31c2a5add282033e29b32 not found: ID does not exist"
Oct 30 21:43:18.463200 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:18.463113    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-969ld"]
Oct 30 21:43:18.467019 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:18.466992    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-969ld"]
Oct 30 21:43:18.477096 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:18.477063    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-qzpg5"]
Oct 30 21:43:18.512074 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:43:18.512042    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7de093c_a605_4406_a940_e3e59abcb9a0.slice/crio-a8bb2f122cf40e98e6856a1f7ce4feeb8a698c73f0064c824d18919d3a3883b3 WatchSource:0}: Error finding container a8bb2f122cf40e98e6856a1f7ce4feeb8a698c73f0064c824d18919d3a3883b3: Status 404 returned error can't find the container with id a8bb2f122cf40e98e6856a1f7ce4feeb8a698c73f0064c824d18919d3a3883b3
Oct 30 21:43:18.779063 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:18.778948    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:19.314964 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:19.314929    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-qzpg5" event={"ID":"b7de093c-a605-4406-a940-e3e59abcb9a0","Type":"ContainerStarted","Data":"a8bb2f122cf40e98e6856a1f7ce4feeb8a698c73f0064c824d18919d3a3883b3"}
Oct 30 21:43:19.961700 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:19.961663    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="10913847-1d45-4524-968b-9dc7e37d1f42" path="/var/lib/kubelet/pods/10913847-1d45-4524-968b-9dc7e37d1f42/volumes"
Oct 30 21:43:22.335473 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:22.335436    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-qzpg5" event={"ID":"b7de093c-a605-4406-a940-e3e59abcb9a0","Type":"ContainerStarted","Data":"09120693881cac96828674a1c00cdf9c86520fcffdeb10f07fcbdb02bd1591ca"}
Oct 30 21:43:22.335896 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:22.335506    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-qzpg5"
Oct 30 21:43:22.543328 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:22.543259    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-qzpg5" podStartSLOduration=2.525905377 podStartE2EDuration="5.543226032s" totalImagesPullingTime="3.017320655s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:43:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:43:22.495072344 +0000 UTC m=+344.672560961" watchObservedRunningTime="2026-10-30 21:43:22.543226032 +0000 UTC m=+344.720714653"
Oct 30 21:43:29.649604 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:29.649572    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:32.198128 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:32.198088    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:32.351331 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:32.351291    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:32.942866 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:32.942835    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:35.567040 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:43:35.566993    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:43:43.343348 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:43:43.343315    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-qzpg5"
Oct 30 21:44:05.567041 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:44:05.567001    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:44:14.133826 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:14.133778    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:14.210333 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:14.210287    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-549656c76d-4cnkj"]
Oct 30 21:44:14.221263 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:14.221192    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-549656c76d-4cnkj"
Oct 30 21:44:14.271322 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:14.271280    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-549656c76d-4cnkj"]
Oct 30 21:44:14.293762 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:14.293714    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8eaf900-23b0-4e83-8c47-bc9037607fb1-metrics-certs\") pod \"maas-controller-549656c76d-4cnkj\" (UID: \"a8eaf900-23b0-4e83-8c47-bc9037607fb1\") " pod="opendatahub/maas-controller-549656c76d-4cnkj"
Oct 30 21:44:14.293939 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:14.293799    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a8eaf900-23b0-4e83-8c47-bc9037607fb1-webhook-cert\") pod \"maas-controller-549656c76d-4cnkj\" (UID: \"a8eaf900-23b0-4e83-8c47-bc9037607fb1\") " pod="opendatahub/maas-controller-549656c76d-4cnkj"
Oct 30 21:44:14.293984 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:14.293930    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8qkh\" (UniqueName: \"kubernetes.io/projected/a8eaf900-23b0-4e83-8c47-bc9037607fb1-kube-api-access-r8qkh\") pod \"maas-controller-549656c76d-4cnkj\" (UID: \"a8eaf900-23b0-4e83-8c47-bc9037607fb1\") " pod="opendatahub/maas-controller-549656c76d-4cnkj"
Oct 30 21:44:14.303292 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:14.303225    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:14.309944 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:14.309906    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 30 21:44:14.315893 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:14.315263    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 30 21:44:14.315893 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:14.315328    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 30 21:44:14.315893 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:14.315331    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7g5nv\""
Oct 30 21:44:14.394844 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:14.394740    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8eaf900-23b0-4e83-8c47-bc9037607fb1-metrics-certs\") pod \"maas-controller-549656c76d-4cnkj\" (UID: \"a8eaf900-23b0-4e83-8c47-bc9037607fb1\") " pod="opendatahub/maas-controller-549656c76d-4cnkj"
Oct 30 21:44:14.394844 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:14.394799    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a8eaf900-23b0-4e83-8c47-bc9037607fb1-webhook-cert\") pod \"maas-controller-549656c76d-4cnkj\" (UID: \"a8eaf900-23b0-4e83-8c47-bc9037607fb1\") " pod="opendatahub/maas-controller-549656c76d-4cnkj"
Oct 30 21:44:14.395088 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:14.394893    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8qkh\" (UniqueName: \"kubernetes.io/projected/a8eaf900-23b0-4e83-8c47-bc9037607fb1-kube-api-access-r8qkh\") pod \"maas-controller-549656c76d-4cnkj\" (UID: \"a8eaf900-23b0-4e83-8c47-bc9037607fb1\") " pod="opendatahub/maas-controller-549656c76d-4cnkj"
Oct 30 21:44:14.397362 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:14.397338    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a8eaf900-23b0-4e83-8c47-bc9037607fb1-webhook-cert\") pod \"maas-controller-549656c76d-4cnkj\" (UID: \"a8eaf900-23b0-4e83-8c47-bc9037607fb1\") " pod="opendatahub/maas-controller-549656c76d-4cnkj"
Oct 30 21:44:14.397480 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:14.397398    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8eaf900-23b0-4e83-8c47-bc9037607fb1-metrics-certs\") pod \"maas-controller-549656c76d-4cnkj\" (UID: \"a8eaf900-23b0-4e83-8c47-bc9037607fb1\") " pod="opendatahub/maas-controller-549656c76d-4cnkj"
Oct 30 21:44:14.478355 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:14.478325    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8qkh\" (UniqueName: \"kubernetes.io/projected/a8eaf900-23b0-4e83-8c47-bc9037607fb1-kube-api-access-r8qkh\") pod \"maas-controller-549656c76d-4cnkj\" (UID: \"a8eaf900-23b0-4e83-8c47-bc9037607fb1\") " pod="opendatahub/maas-controller-549656c76d-4cnkj"
Oct 30 21:44:14.535311 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:14.535274    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-549656c76d-4cnkj"
Oct 30 21:44:14.558775 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:14.558741    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:14.735883 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:14.735801    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:14.944546 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:44:14.944514    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8eaf900_23b0_4e83_8c47_bc9037607fb1.slice/crio-8cfb5d9e72b4b8fd7c6fc6dbbd4ce31657fdc6f01eb21a9adbb643c8e07354e6 WatchSource:0}: Error finding container 8cfb5d9e72b4b8fd7c6fc6dbbd4ce31657fdc6f01eb21a9adbb643c8e07354e6: Status 404 returned error can't find the container with id 8cfb5d9e72b4b8fd7c6fc6dbbd4ce31657fdc6f01eb21a9adbb643c8e07354e6
Oct 30 21:44:14.944546 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:14.944537    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-549656c76d-4cnkj"]
Oct 30 21:44:14.964176 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:14.964138    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-84769bb5bf-j5wbn"]
Oct 30 21:44:14.969344 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:14.969313    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84769bb5bf-j5wbn"
Oct 30 21:44:14.999511 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:14.999427    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/888e372b-f344-412f-9f59-6411a4f1177e-metrics-certs\") pod \"maas-controller-84769bb5bf-j5wbn\" (UID: \"888e372b-f344-412f-9f59-6411a4f1177e\") " pod="opendatahub/maas-controller-84769bb5bf-j5wbn"
Oct 30 21:44:14.999511 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:14.999478    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpdkq\" (UniqueName: \"kubernetes.io/projected/888e372b-f344-412f-9f59-6411a4f1177e-kube-api-access-jpdkq\") pod \"maas-controller-84769bb5bf-j5wbn\" (UID: \"888e372b-f344-412f-9f59-6411a4f1177e\") " pod="opendatahub/maas-controller-84769bb5bf-j5wbn"
Oct 30 21:44:14.999716 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:14.999632    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/888e372b-f344-412f-9f59-6411a4f1177e-webhook-cert\") pod \"maas-controller-84769bb5bf-j5wbn\" (UID: \"888e372b-f344-412f-9f59-6411a4f1177e\") " pod="opendatahub/maas-controller-84769bb5bf-j5wbn"
Oct 30 21:44:15.058702 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:15.058671    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84769bb5bf-j5wbn"]
Oct 30 21:44:15.101203 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:15.101153    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/888e372b-f344-412f-9f59-6411a4f1177e-webhook-cert\") pod \"maas-controller-84769bb5bf-j5wbn\" (UID: \"888e372b-f344-412f-9f59-6411a4f1177e\") " pod="opendatahub/maas-controller-84769bb5bf-j5wbn"
Oct 30 21:44:15.101388 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:15.101251    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/888e372b-f344-412f-9f59-6411a4f1177e-metrics-certs\") pod \"maas-controller-84769bb5bf-j5wbn\" (UID: \"888e372b-f344-412f-9f59-6411a4f1177e\") " pod="opendatahub/maas-controller-84769bb5bf-j5wbn"
Oct 30 21:44:15.101388 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:15.101288    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpdkq\" (UniqueName: \"kubernetes.io/projected/888e372b-f344-412f-9f59-6411a4f1177e-kube-api-access-jpdkq\") pod \"maas-controller-84769bb5bf-j5wbn\" (UID: \"888e372b-f344-412f-9f59-6411a4f1177e\") " pod="opendatahub/maas-controller-84769bb5bf-j5wbn"
Oct 30 21:44:15.103815 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:15.103783    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/888e372b-f344-412f-9f59-6411a4f1177e-webhook-cert\") pod \"maas-controller-84769bb5bf-j5wbn\" (UID: \"888e372b-f344-412f-9f59-6411a4f1177e\") " pod="opendatahub/maas-controller-84769bb5bf-j5wbn"
Oct 30 21:44:15.103815 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:15.103810    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/888e372b-f344-412f-9f59-6411a4f1177e-metrics-certs\") pod \"maas-controller-84769bb5bf-j5wbn\" (UID: \"888e372b-f344-412f-9f59-6411a4f1177e\") " pod="opendatahub/maas-controller-84769bb5bf-j5wbn"
Oct 30 21:44:15.162207 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:15.162169    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpdkq\" (UniqueName: \"kubernetes.io/projected/888e372b-f344-412f-9f59-6411a4f1177e-kube-api-access-jpdkq\") pod \"maas-controller-84769bb5bf-j5wbn\" (UID: \"888e372b-f344-412f-9f59-6411a4f1177e\") " pod="opendatahub/maas-controller-84769bb5bf-j5wbn"
Oct 30 21:44:15.283606 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:15.283521    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84769bb5bf-j5wbn"
Oct 30 21:44:15.638102 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:15.638067    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-549656c76d-4cnkj" event={"ID":"a8eaf900-23b0-4e83-8c47-bc9037607fb1","Type":"ContainerStarted","Data":"8cfb5d9e72b4b8fd7c6fc6dbbd4ce31657fdc6f01eb21a9adbb643c8e07354e6"}
Oct 30 21:44:15.653840 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:15.653813    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84769bb5bf-j5wbn"]
Oct 30 21:44:15.655206 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:44:15.655176    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod888e372b_f344_412f_9f59_6411a4f1177e.slice/crio-5722d79e2dba1b83b08ca726a2ad7c9dc0918f93db425e5f2616a4986e3ff725 WatchSource:0}: Error finding container 5722d79e2dba1b83b08ca726a2ad7c9dc0918f93db425e5f2616a4986e3ff725: Status 404 returned error can't find the container with id 5722d79e2dba1b83b08ca726a2ad7c9dc0918f93db425e5f2616a4986e3ff725
Oct 30 21:44:16.644433 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:16.644382    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84769bb5bf-j5wbn" event={"ID":"888e372b-f344-412f-9f59-6411a4f1177e","Type":"ContainerStarted","Data":"5722d79e2dba1b83b08ca726a2ad7c9dc0918f93db425e5f2616a4986e3ff725"}
Oct 30 21:44:19.446208 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:19.446115    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-l8t6z"]
Oct 30 21:44:19.450016 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:19.449988    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-l8t6z"
Oct 30 21:44:19.500469 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:19.500363    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Oct 30 21:44:19.500469 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:19.500401    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Oct 30 21:44:19.500469 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:19.500441    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Oct 30 21:44:19.546390 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:19.546355    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-66p9x\""
Oct 30 21:44:19.548733 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:19.548408    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1817be16-1033-45eb-8499-5c2d201bbc2e-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-l8t6z\" (UID: \"1817be16-1033-45eb-8499-5c2d201bbc2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-l8t6z"
Oct 30 21:44:19.548733 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:19.548453    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1817be16-1033-45eb-8499-5c2d201bbc2e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-l8t6z\" (UID: \"1817be16-1033-45eb-8499-5c2d201bbc2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-l8t6z"
Oct 30 21:44:19.548733 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:19.548479    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1817be16-1033-45eb-8499-5c2d201bbc2e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-l8t6z\" (UID: \"1817be16-1033-45eb-8499-5c2d201bbc2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-l8t6z"
Oct 30 21:44:19.548733 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:19.548509    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24bxv\" (UniqueName: \"kubernetes.io/projected/1817be16-1033-45eb-8499-5c2d201bbc2e-kube-api-access-24bxv\") pod \"istiod-openshift-gateway-7ddd9cfdc8-l8t6z\" (UID: \"1817be16-1033-45eb-8499-5c2d201bbc2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-l8t6z"
Oct 30 21:44:19.548733 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:19.548534    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1817be16-1033-45eb-8499-5c2d201bbc2e-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-l8t6z\" (UID: \"1817be16-1033-45eb-8499-5c2d201bbc2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-l8t6z"
Oct 30 21:44:19.548733 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:19.548589    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1817be16-1033-45eb-8499-5c2d201bbc2e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-l8t6z\" (UID: \"1817be16-1033-45eb-8499-5c2d201bbc2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-l8t6z"
Oct 30 21:44:19.548733 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:19.548620    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1817be16-1033-45eb-8499-5c2d201bbc2e-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-l8t6z\" (UID: \"1817be16-1033-45eb-8499-5c2d201bbc2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-l8t6z"
Oct 30 21:44:19.551475 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:19.551450    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-l8t6z"]
Oct 30 21:44:19.649705 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:19.649663    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1817be16-1033-45eb-8499-5c2d201bbc2e-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-l8t6z\" (UID: \"1817be16-1033-45eb-8499-5c2d201bbc2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-l8t6z"
Oct 30 21:44:19.649705 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:19.649706    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1817be16-1033-45eb-8499-5c2d201bbc2e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-l8t6z\" (UID: \"1817be16-1033-45eb-8499-5c2d201bbc2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-l8t6z"
Oct 30 21:44:19.649983 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:19.649722    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1817be16-1033-45eb-8499-5c2d201bbc2e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-l8t6z\" (UID: \"1817be16-1033-45eb-8499-5c2d201bbc2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-l8t6z"
Oct 30 21:44:19.649983 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:19.649865    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24bxv\" (UniqueName: \"kubernetes.io/projected/1817be16-1033-45eb-8499-5c2d201bbc2e-kube-api-access-24bxv\") pod \"istiod-openshift-gateway-7ddd9cfdc8-l8t6z\" (UID: \"1817be16-1033-45eb-8499-5c2d201bbc2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-l8t6z"
Oct 30 21:44:19.649983 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:19.649911    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1817be16-1033-45eb-8499-5c2d201bbc2e-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-l8t6z\" (UID: \"1817be16-1033-45eb-8499-5c2d201bbc2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-l8t6z"
Oct 30 21:44:19.649983 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:19.649980    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1817be16-1033-45eb-8499-5c2d201bbc2e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-l8t6z\" (UID: \"1817be16-1033-45eb-8499-5c2d201bbc2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-l8t6z"
Oct 30 21:44:19.650191 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:19.650018    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1817be16-1033-45eb-8499-5c2d201bbc2e-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-l8t6z\" (UID: \"1817be16-1033-45eb-8499-5c2d201bbc2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-l8t6z"
Oct 30 21:44:19.650754 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:19.650729    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1817be16-1033-45eb-8499-5c2d201bbc2e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-l8t6z\" (UID: \"1817be16-1033-45eb-8499-5c2d201bbc2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-l8t6z"
Oct 30 21:44:19.652174 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:19.652151    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1817be16-1033-45eb-8499-5c2d201bbc2e-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-l8t6z\" (UID: \"1817be16-1033-45eb-8499-5c2d201bbc2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-l8t6z"
Oct 30 21:44:19.652301 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:19.652226    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1817be16-1033-45eb-8499-5c2d201bbc2e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-l8t6z\" (UID: \"1817be16-1033-45eb-8499-5c2d201bbc2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-l8t6z"
Oct 30 21:44:19.652548 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:19.652526    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1817be16-1033-45eb-8499-5c2d201bbc2e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-l8t6z\" (UID: \"1817be16-1033-45eb-8499-5c2d201bbc2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-l8t6z"
Oct 30 21:44:19.652611 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:19.652576    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1817be16-1033-45eb-8499-5c2d201bbc2e-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-l8t6z\" (UID: \"1817be16-1033-45eb-8499-5c2d201bbc2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-l8t6z"
Oct 30 21:44:19.708620 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:19.708545    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1817be16-1033-45eb-8499-5c2d201bbc2e-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-l8t6z\" (UID: \"1817be16-1033-45eb-8499-5c2d201bbc2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-l8t6z"
Oct 30 21:44:19.708818 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:19.708795    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-24bxv\" (UniqueName: \"kubernetes.io/projected/1817be16-1033-45eb-8499-5c2d201bbc2e-kube-api-access-24bxv\") pod \"istiod-openshift-gateway-7ddd9cfdc8-l8t6z\" (UID: \"1817be16-1033-45eb-8499-5c2d201bbc2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-l8t6z"
Oct 30 21:44:19.765466 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:19.765427    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-l8t6z"
Oct 30 21:44:20.184743 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:20.184710    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-l8t6z"]
Oct 30 21:44:20.186285 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:44:20.186257    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1817be16_1033_45eb_8499_5c2d201bbc2e.slice/crio-b6adb98d798478299272165d2496c2d60b4d6afbdedd1438604227bfe5c73ebe WatchSource:0}: Error finding container b6adb98d798478299272165d2496c2d60b4d6afbdedd1438604227bfe5c73ebe: Status 404 returned error can't find the container with id b6adb98d798478299272165d2496c2d60b4d6afbdedd1438604227bfe5c73ebe
Oct 30 21:44:20.674571 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:20.674537    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-l8t6z" event={"ID":"1817be16-1033-45eb-8499-5c2d201bbc2e","Type":"ContainerStarted","Data":"b6adb98d798478299272165d2496c2d60b4d6afbdedd1438604227bfe5c73ebe"}
Oct 30 21:44:23.239743 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:23.239702    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:23.401860 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:23.401811    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:23.697504 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:23.697446    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-549656c76d-4cnkj" event={"ID":"a8eaf900-23b0-4e83-8c47-bc9037607fb1","Type":"ContainerStarted","Data":"5799527dc592a3fff672ccd9c3f19be5cce5ba534b352e491208558b5c35cca9"}
Oct 30 21:44:23.697712 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:23.697691    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-549656c76d-4cnkj"
Oct 30 21:44:23.699673 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:23.699636    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84769bb5bf-j5wbn" event={"ID":"888e372b-f344-412f-9f59-6411a4f1177e","Type":"ContainerStarted","Data":"4f3c6eac6fc63a3b29df36fbc8fd55bcfdc632fcfb6c332498b99278901b9924"}
Oct 30 21:44:23.699819 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:23.699776    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-84769bb5bf-j5wbn"
Oct 30 21:44:23.811660 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:23.811593    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-549656c76d-4cnkj" podStartSLOduration=1.908419206 podStartE2EDuration="9.811574638s" totalImagesPullingTime="7.903155432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:44:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:44:23.809792416 +0000 UTC m=+405.987281034" watchObservedRunningTime="2026-10-30 21:44:23.811574638 +0000 UTC m=+405.989063257"
Oct 30 21:44:23.953298 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:23.953172    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-84769bb5bf-j5wbn" podStartSLOduration=2.809645405 podStartE2EDuration="9.953156387s" totalImagesPullingTime="7.143510982s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:44:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:44:23.951655321 +0000 UTC m=+406.129144014" watchObservedRunningTime="2026-10-30 21:44:23.953156387 +0000 UTC m=+406.130645004"
Oct 30 21:44:23.975891 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:23.975848    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:24.124205 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:24.124168    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:24.261898 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:24.261865    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:44:24.262213 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:24.261960    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:44:24.297612 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:24.297580    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:24.446680 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:24.446637    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:24.609231 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:24.609127    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:24.707600 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:24.707563    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-l8t6z" event={"ID":"1817be16-1033-45eb-8499-5c2d201bbc2e","Type":"ContainerStarted","Data":"b78685a7df9182774be3a27757d0f0e9615c010a0ba1d91076d5c72f8daf6524"}
Oct 30 21:44:24.707869 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:24.707849    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-l8t6z"
Oct 30 21:44:24.765678 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:24.765645    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:24.830651 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:24.830589    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-l8t6z" podStartSLOduration=1.7580008 podStartE2EDuration="5.8305733s" totalImagesPullingTime="4.0725725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:44:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:44:24.827782517 +0000 UTC m=+407.005271137" watchObservedRunningTime="2026-10-30 21:44:24.8305733 +0000 UTC m=+407.008061917"
Oct 30 21:44:24.918655 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:24.918617    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:25.073916 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:25.073880    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:25.714498 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:25.714468    4871 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-l8t6z container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:44:25.714946 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:25.714523    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-l8t6z" podUID="1817be16-1033-45eb-8499-5c2d201bbc2e" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:44:26.167492 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:26.167416    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:28.138049 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:28.138014    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:28.303584 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:28.303552    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:28.468311 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:28.468210    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:28.713898 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:28.713864    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-l8t6z"
Oct 30 21:44:30.078747 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:30.078706    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:31.218274 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:31.218217    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:31.390348 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:31.390314    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:31.595531 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:31.595439    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:31.797966 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:31.797926    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:32.048052 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:32.048019    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:32.212743 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:32.212710    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:32.376356 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:32.376319    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:34.710141 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:34.710109    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-549656c76d-4cnkj"
Oct 30 21:44:34.710574 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:34.710186    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-84769bb5bf-j5wbn"
Oct 30 21:44:34.975201 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:34.975110    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-549656c76d-4cnkj"]
Oct 30 21:44:34.975514 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:34.975455    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-549656c76d-4cnkj" podUID="a8eaf900-23b0-4e83-8c47-bc9037607fb1" containerName="manager" containerID="cri-o://5799527dc592a3fff672ccd9c3f19be5cce5ba534b352e491208558b5c35cca9" gracePeriod=30
Oct 30 21:44:35.227655 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:35.227591    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-549656c76d-4cnkj"
Oct 30 21:44:35.299507 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:35.299471    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a8eaf900-23b0-4e83-8c47-bc9037607fb1-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8eaf900-23b0-4e83-8c47-bc9037607fb1-metrics-certs\") pod \"a8eaf900-23b0-4e83-8c47-bc9037607fb1\" (UID: \"a8eaf900-23b0-4e83-8c47-bc9037607fb1\") "
Oct 30 21:44:35.299507 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:35.299513    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a8eaf900-23b0-4e83-8c47-bc9037607fb1-kube-api-access-r8qkh\" (UniqueName: \"kubernetes.io/projected/a8eaf900-23b0-4e83-8c47-bc9037607fb1-kube-api-access-r8qkh\") pod \"a8eaf900-23b0-4e83-8c47-bc9037607fb1\" (UID: \"a8eaf900-23b0-4e83-8c47-bc9037607fb1\") "
Oct 30 21:44:35.299753 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:35.299569    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a8eaf900-23b0-4e83-8c47-bc9037607fb1-webhook-cert\" (UniqueName: \"kubernetes.io/secret/a8eaf900-23b0-4e83-8c47-bc9037607fb1-webhook-cert\") pod \"a8eaf900-23b0-4e83-8c47-bc9037607fb1\" (UID: \"a8eaf900-23b0-4e83-8c47-bc9037607fb1\") "
Oct 30 21:44:35.301890 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:35.301861    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8eaf900-23b0-4e83-8c47-bc9037607fb1-kube-api-access-r8qkh" pod "a8eaf900-23b0-4e83-8c47-bc9037607fb1" (UID: "a8eaf900-23b0-4e83-8c47-bc9037607fb1"). InnerVolumeSpecName "kube-api-access-r8qkh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:44:35.301985 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:35.301880    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8eaf900-23b0-4e83-8c47-bc9037607fb1-metrics-certs" pod "a8eaf900-23b0-4e83-8c47-bc9037607fb1" (UID: "a8eaf900-23b0-4e83-8c47-bc9037607fb1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:44:35.301985 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:35.301927    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8eaf900-23b0-4e83-8c47-bc9037607fb1-webhook-cert" pod "a8eaf900-23b0-4e83-8c47-bc9037607fb1" (UID: "a8eaf900-23b0-4e83-8c47-bc9037607fb1"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:44:35.400505 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:35.400464    4871 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a8eaf900-23b0-4e83-8c47-bc9037607fb1-webhook-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:44:35.400505 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:35.400498    4871 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8eaf900-23b0-4e83-8c47-bc9037607fb1-metrics-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:44:35.400505 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:35.400510    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8qkh\" (UniqueName: \"kubernetes.io/projected/a8eaf900-23b0-4e83-8c47-bc9037607fb1-kube-api-access-r8qkh\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:44:35.568713 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:44:35.568610    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:44:35.773992 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:35.773956    4871 generic.go:356] "Generic (PLEG): container finished" podID="a8eaf900-23b0-4e83-8c47-bc9037607fb1" containerID="5799527dc592a3fff672ccd9c3f19be5cce5ba534b352e491208558b5c35cca9" exitCode=0
Oct 30 21:44:35.774522 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:35.774004    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-549656c76d-4cnkj"
Oct 30 21:44:35.774522 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:35.774038    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-549656c76d-4cnkj" event={"ID":"a8eaf900-23b0-4e83-8c47-bc9037607fb1","Type":"ContainerDied","Data":"5799527dc592a3fff672ccd9c3f19be5cce5ba534b352e491208558b5c35cca9"}
Oct 30 21:44:35.774522 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:35.774098    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-549656c76d-4cnkj" event={"ID":"a8eaf900-23b0-4e83-8c47-bc9037607fb1","Type":"ContainerDied","Data":"8cfb5d9e72b4b8fd7c6fc6dbbd4ce31657fdc6f01eb21a9adbb643c8e07354e6"}
Oct 30 21:44:35.774522 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:35.774126    4871 scope.go:122] "RemoveContainer" containerID="5799527dc592a3fff672ccd9c3f19be5cce5ba534b352e491208558b5c35cca9"
Oct 30 21:44:35.783969 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:35.783949    4871 scope.go:122] "RemoveContainer" containerID="5799527dc592a3fff672ccd9c3f19be5cce5ba534b352e491208558b5c35cca9"
Oct 30 21:44:35.784334 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:44:35.784312    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5799527dc592a3fff672ccd9c3f19be5cce5ba534b352e491208558b5c35cca9\": container with ID starting with 5799527dc592a3fff672ccd9c3f19be5cce5ba534b352e491208558b5c35cca9 not found: ID does not exist" containerID="5799527dc592a3fff672ccd9c3f19be5cce5ba534b352e491208558b5c35cca9"
Oct 30 21:44:35.784387 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:35.784347    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5799527dc592a3fff672ccd9c3f19be5cce5ba534b352e491208558b5c35cca9"} err="failed to get container status \"5799527dc592a3fff672ccd9c3f19be5cce5ba534b352e491208558b5c35cca9\": rpc error: code = NotFound desc = could not find container \"5799527dc592a3fff672ccd9c3f19be5cce5ba534b352e491208558b5c35cca9\": container with ID starting with 5799527dc592a3fff672ccd9c3f19be5cce5ba534b352e491208558b5c35cca9 not found: ID does not exist"
Oct 30 21:44:35.937691 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:35.937654    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-549656c76d-4cnkj"]
Oct 30 21:44:35.940615 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:35.940591    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-549656c76d-4cnkj"]
Oct 30 21:44:35.959024 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:35.958990    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a8eaf900-23b0-4e83-8c47-bc9037607fb1" path="/var/lib/kubelet/pods/a8eaf900-23b0-4e83-8c47-bc9037607fb1/volumes"
Oct 30 21:44:56.940008 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:56.939963    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:57.112048 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:57.112011    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:58.949246 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:58.949197    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:59.156966 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:59.156913    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:59.313386 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:59.313303    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:59.475903 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:44:59.475871    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:05.579144 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:45:05.579103    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:45:13.180412 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:45:13.180368    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-hkg2v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:45:13.180814 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:45:13.180467    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v" podUID="2c5a0598-58b9-4032-8e23-27daf0444149" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:13.578386 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:45:13.578304    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:13.815729 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:45:13.815685    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:15.000772 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:45:15.000731    4871 generic.go:356] "Generic (PLEG): container finished" podID="2c5a0598-58b9-4032-8e23-27daf0444149" containerID="4c2b14109615eae580160381e381270bccd8598b240e30f91c9ee2f4c7961fa3" exitCode=0
Oct 30 21:45:15.000772 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:45:15.000754    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v" event={"ID":"2c5a0598-58b9-4032-8e23-27daf0444149","Type":"ContainerDied","Data":"4c2b14109615eae580160381e381270bccd8598b240e30f91c9ee2f4c7961fa3"}
Oct 30 21:45:15.001291 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:45:15.001150    4871 scope.go:122] "RemoveContainer" containerID="4c2b14109615eae580160381e381270bccd8598b240e30f91c9ee2f4c7961fa3"
Oct 30 21:45:15.002477 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:45:15.002441    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:45:15.002575 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:45:15.002520    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:45:15.002575 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:45:15.002556    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:45:16.012914 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:45:16.012865    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v" event={"ID":"2c5a0598-58b9-4032-8e23-27daf0444149","Type":"ContainerStarted","Data":"b3f280e29014840e3e939e906b261a2640c278f1f407db4b98ddffd805301b73"}
Oct 30 21:45:17.002037 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:45:17.002002    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"
Oct 30 21:45:17.002406 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:45:17.002376    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-hkg2v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Oct 30 21:45:17.002632 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:45:17.002605    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v" podUID="2c5a0598-58b9-4032-8e23-27daf0444149" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Oct 30 21:45:18.003285 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:45:18.002655    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-hkg2v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Oct 30 21:45:18.003285 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:45:18.002741    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v" podUID="2c5a0598-58b9-4032-8e23-27daf0444149" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Oct 30 21:45:19.006261 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:45:19.002688    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-hkg2v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Oct 30 21:45:19.006261 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:45:19.002764    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v" podUID="2c5a0598-58b9-4032-8e23-27daf0444149" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Oct 30 21:45:20.008416 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:45:20.008374    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-hkg2v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Oct 30 21:45:20.009027 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:45:20.008994    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v" podUID="2c5a0598-58b9-4032-8e23-27daf0444149" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Oct 30 21:45:21.002352 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:45:21.002316    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-hkg2v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Oct 30 21:45:21.002550 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:45:21.002384    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v" podUID="2c5a0598-58b9-4032-8e23-27daf0444149" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Oct 30 21:45:22.005401 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:45:22.005365    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"
Oct 30 21:45:22.005894 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:45:22.005769    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"
Oct 30 21:45:22.006495 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:45:22.006477    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"
Oct 30 21:45:27.906427 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:45:27.906393    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:30.349799 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:45:30.349765    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:35.569358 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:45:35.569316    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:45:36.133333 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:45:36.133294    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:36.405157 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:45:36.405054    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:05.568052 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:46:05.567963    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:46:13.691791 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:46:13.691757    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:13.941515 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:46:13.941476    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:20.534907 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:46:20.534869    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:20.786948 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:46:20.786863    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:23.936416 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:46:23.936381    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:26.201462 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:46:26.201424    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:30.725949 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:46:30.725905    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:34.632931 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:46:34.632892    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:35.568190 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:46:35.568155    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:46:43.314774 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:46:43.314724    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:43.824268 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:46:43.824206    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:45.278793 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:46:45.278755    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:45.599575 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:46:45.599487    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:52.221535 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:46:52.221446    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:02.166414 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:02.166372    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:05.570027 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:47:05.569986    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:47:05.679103 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:05.679056    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:08.615462 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:08.615428    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:09.658784 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:09.658748    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:09.858364 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:09.858333    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:12.089437 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:12.089404    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:15.639747 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:15.639697    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:15.822811 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:15.822773    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:19.610475 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:19.610375    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-79krz"]
Oct 30 21:47:19.611067 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:19.610852    4871 cpu_manager.go:414] "containerMap: removing container" podUID="a8eaf900-23b0-4e83-8c47-bc9037607fb1" containerName="manager"
Oct 30 21:47:19.611067 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:19.610873    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a8eaf900-23b0-4e83-8c47-bc9037607fb1" containerName="manager"
Oct 30 21:47:19.611067 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:19.611017    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="a8eaf900-23b0-4e83-8c47-bc9037607fb1" containerName="manager"
Oct 30 21:47:19.614586 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:19.614554    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-79krz"
Oct 30 21:47:19.673443 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:19.673405    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-79krz"]
Oct 30 21:47:19.690821 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:19.690787    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Oct 30 21:47:19.690821 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:19.690805    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Oct 30 21:47:19.702969 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:19.702936    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0a4171a-a8b7-45be-8bcc-8bc8d598e091-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-79krz\" (UID: \"e0a4171a-a8b7-45be-8bcc-8bc8d598e091\") " pod="llm/echo-simulator-kserve-64b6f8f96c-79krz"
Oct 30 21:47:19.703291 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:19.702985    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lljcb\" (UniqueName: \"kubernetes.io/projected/e0a4171a-a8b7-45be-8bcc-8bc8d598e091-kube-api-access-lljcb\") pod \"echo-simulator-kserve-64b6f8f96c-79krz\" (UID: \"e0a4171a-a8b7-45be-8bcc-8bc8d598e091\") " pod="llm/echo-simulator-kserve-64b6f8f96c-79krz"
Oct 30 21:47:19.703291 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:19.703119    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0a4171a-a8b7-45be-8bcc-8bc8d598e091-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-79krz\" (UID: \"e0a4171a-a8b7-45be-8bcc-8bc8d598e091\") " pod="llm/echo-simulator-kserve-64b6f8f96c-79krz"
Oct 30 21:47:19.703291 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:19.703155    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0a4171a-a8b7-45be-8bcc-8bc8d598e091-home\") pod \"echo-simulator-kserve-64b6f8f96c-79krz\" (UID: \"e0a4171a-a8b7-45be-8bcc-8bc8d598e091\") " pod="llm/echo-simulator-kserve-64b6f8f96c-79krz"
Oct 30 21:47:19.703291 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:19.703179    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0a4171a-a8b7-45be-8bcc-8bc8d598e091-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-79krz\" (UID: \"e0a4171a-a8b7-45be-8bcc-8bc8d598e091\") " pod="llm/echo-simulator-kserve-64b6f8f96c-79krz"
Oct 30 21:47:19.703568 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:19.703543    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0a4171a-a8b7-45be-8bcc-8bc8d598e091-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-79krz\" (UID: \"e0a4171a-a8b7-45be-8bcc-8bc8d598e091\") " pod="llm/echo-simulator-kserve-64b6f8f96c-79krz"
Oct 30 21:47:19.717173 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:19.717146    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-brncp\""
Oct 30 21:47:19.717377 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:19.717184    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"echo-simulator-kserve-self-signed-certs\""
Oct 30 21:47:19.739752 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:19.739720    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:19.806138 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:19.806077    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0a4171a-a8b7-45be-8bcc-8bc8d598e091-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-79krz\" (UID: \"e0a4171a-a8b7-45be-8bcc-8bc8d598e091\") " pod="llm/echo-simulator-kserve-64b6f8f96c-79krz"
Oct 30 21:47:19.806376 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:19.806149    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lljcb\" (UniqueName: \"kubernetes.io/projected/e0a4171a-a8b7-45be-8bcc-8bc8d598e091-kube-api-access-lljcb\") pod \"echo-simulator-kserve-64b6f8f96c-79krz\" (UID: \"e0a4171a-a8b7-45be-8bcc-8bc8d598e091\") " pod="llm/echo-simulator-kserve-64b6f8f96c-79krz"
Oct 30 21:47:19.806376 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:19.806208    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0a4171a-a8b7-45be-8bcc-8bc8d598e091-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-79krz\" (UID: \"e0a4171a-a8b7-45be-8bcc-8bc8d598e091\") " pod="llm/echo-simulator-kserve-64b6f8f96c-79krz"
Oct 30 21:47:19.806376 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:19.806253    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0a4171a-a8b7-45be-8bcc-8bc8d598e091-home\") pod \"echo-simulator-kserve-64b6f8f96c-79krz\" (UID: \"e0a4171a-a8b7-45be-8bcc-8bc8d598e091\") " pod="llm/echo-simulator-kserve-64b6f8f96c-79krz"
Oct 30 21:47:19.806376 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:19.806283    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0a4171a-a8b7-45be-8bcc-8bc8d598e091-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-79krz\" (UID: \"e0a4171a-a8b7-45be-8bcc-8bc8d598e091\") " pod="llm/echo-simulator-kserve-64b6f8f96c-79krz"
Oct 30 21:47:19.806376 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:19.806325    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0a4171a-a8b7-45be-8bcc-8bc8d598e091-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-79krz\" (UID: \"e0a4171a-a8b7-45be-8bcc-8bc8d598e091\") " pod="llm/echo-simulator-kserve-64b6f8f96c-79krz"
Oct 30 21:47:19.806849 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:19.806825    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0a4171a-a8b7-45be-8bcc-8bc8d598e091-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-79krz\" (UID: \"e0a4171a-a8b7-45be-8bcc-8bc8d598e091\") " pod="llm/echo-simulator-kserve-64b6f8f96c-79krz"
Oct 30 21:47:19.807118 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:19.807098    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0a4171a-a8b7-45be-8bcc-8bc8d598e091-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-79krz\" (UID: \"e0a4171a-a8b7-45be-8bcc-8bc8d598e091\") " pod="llm/echo-simulator-kserve-64b6f8f96c-79krz"
Oct 30 21:47:19.807615 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:19.807565    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0a4171a-a8b7-45be-8bcc-8bc8d598e091-home\") pod \"echo-simulator-kserve-64b6f8f96c-79krz\" (UID: \"e0a4171a-a8b7-45be-8bcc-8bc8d598e091\") " pod="llm/echo-simulator-kserve-64b6f8f96c-79krz"
Oct 30 21:47:19.810335 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:19.810308    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0a4171a-a8b7-45be-8bcc-8bc8d598e091-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-79krz\" (UID: \"e0a4171a-a8b7-45be-8bcc-8bc8d598e091\") " pod="llm/echo-simulator-kserve-64b6f8f96c-79krz"
Oct 30 21:47:19.810440 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:19.810339    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0a4171a-a8b7-45be-8bcc-8bc8d598e091-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-79krz\" (UID: \"e0a4171a-a8b7-45be-8bcc-8bc8d598e091\") " pod="llm/echo-simulator-kserve-64b6f8f96c-79krz"
Oct 30 21:47:19.866603 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:19.866566    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lljcb\" (UniqueName: \"kubernetes.io/projected/e0a4171a-a8b7-45be-8bcc-8bc8d598e091-kube-api-access-lljcb\") pod \"echo-simulator-kserve-64b6f8f96c-79krz\" (UID: \"e0a4171a-a8b7-45be-8bcc-8bc8d598e091\") " pod="llm/echo-simulator-kserve-64b6f8f96c-79krz"
Oct 30 21:47:19.933885 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:19.933827    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-79krz"
Oct 30 21:47:20.342449 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:20.342420    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-79krz"]
Oct 30 21:47:20.344692 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:47:20.344660    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0a4171a_a8b7_45be_8bcc_8bc8d598e091.slice/crio-faa3df44feedc6ef47ea3340967c9004de9d6f2eb7b309077b82d6cdd616ab22 WatchSource:0}: Error finding container faa3df44feedc6ef47ea3340967c9004de9d6f2eb7b309077b82d6cdd616ab22: Status 404 returned error can't find the container with id faa3df44feedc6ef47ea3340967c9004de9d6f2eb7b309077b82d6cdd616ab22
Oct 30 21:47:20.723785 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:20.723743    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-79krz" event={"ID":"e0a4171a-a8b7-45be-8bcc-8bc8d598e091","Type":"ContainerStarted","Data":"faa3df44feedc6ef47ea3340967c9004de9d6f2eb7b309077b82d6cdd616ab22"}
Oct 30 21:47:23.148972 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:23.148937    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:23.746836 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:23.746794    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-79krz" event={"ID":"e0a4171a-a8b7-45be-8bcc-8bc8d598e091","Type":"ContainerStarted","Data":"a701b66b222ddfb811cf3baef6290997d5aed081c9632f2ae4703aedf353f37c"}
Oct 30 21:47:23.934063 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:23.934027    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/echo-simulator-kserve-64b6f8f96c-79krz"
Oct 30 21:47:26.366675 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:26.366638    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:29.933998 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:29.933961    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/echo-simulator-kserve-64b6f8f96c-79krz"
Oct 30 21:47:29.950895 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:29.950866    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/echo-simulator-kserve-64b6f8f96c-79krz"
Oct 30 21:47:30.069120 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:30.069062    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/echo-simulator-kserve-64b6f8f96c-79krz" podStartSLOduration=8.331236914 podStartE2EDuration="11.069048251s" totalImagesPullingTime="2.737811337s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:47:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:47:23.877196909 +0000 UTC m=+586.054685523" watchObservedRunningTime="2026-10-30 21:47:30.069048251 +0000 UTC m=+592.246536866"
Oct 30 21:47:30.798596 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:30.798566    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/echo-simulator-kserve-64b6f8f96c-79krz"
Oct 30 21:47:31.027613 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:31.027571    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:35.570752 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:47:35.570703    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:47:38.027290 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:38.027261    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx_8d401654-8a4c-4a79-b2d3-612765312421/extract/0.log"
Oct 30 21:47:38.028699 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:38.028671    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx_8d401654-8a4c-4a79-b2d3-612765312421/extract/0.log"
Oct 30 21:47:38.042280 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:38.041644    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/2.log"
Oct 30 21:47:38.043030 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:38.043009    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/2.log"
Oct 30 21:47:38.049351 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:38.049333    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzntd_2360fddc-52a5-42a5-82ca-a05aff84d217/ovn-acl-logging/0.log"
Oct 30 21:47:38.050529 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:38.050509    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzntd_2360fddc-52a5-42a5-82ca-a05aff84d217/ovn-acl-logging/0.log"
Oct 30 21:47:44.085773 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:44.085737    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn"]
Oct 30 21:47:44.089677 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:44.089655    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn"
Oct 30 21:47:44.214759 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:44.214719    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d-istio-envoy\") pod \"maas-default-gateway-openshift-default-7df854bf67-vjrqn\" (UID: \"cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn"
Oct 30 21:47:44.214953 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:44.214772    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d-workload-certs\") pod \"maas-default-gateway-openshift-default-7df854bf67-vjrqn\" (UID: \"cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn"
Oct 30 21:47:44.214953 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:44.214832    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d-workload-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-vjrqn\" (UID: \"cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn"
Oct 30 21:47:44.214953 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:44.214850    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d-credential-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-vjrqn\" (UID: \"cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn"
Oct 30 21:47:44.215094 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:44.214957    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d-istio-token\") pod \"maas-default-gateway-openshift-default-7df854bf67-vjrqn\" (UID: \"cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn"
Oct 30 21:47:44.215094 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:44.215055    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d-istio-data\") pod \"maas-default-gateway-openshift-default-7df854bf67-vjrqn\" (UID: \"cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn"
Oct 30 21:47:44.215162 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:44.215141    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7df854bf67-vjrqn\" (UID: \"cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn"
Oct 30 21:47:44.215199 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:44.215171    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7df854bf67-vjrqn\" (UID: \"cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn"
Oct 30 21:47:44.215276 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:44.215197    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdlqs\" (UniqueName: \"kubernetes.io/projected/cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d-kube-api-access-mdlqs\") pod \"maas-default-gateway-openshift-default-7df854bf67-vjrqn\" (UID: \"cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn"
Oct 30 21:47:44.296974 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:44.296933    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn"]
Oct 30 21:47:44.316102 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:44.316064    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d-istio-envoy\") pod \"maas-default-gateway-openshift-default-7df854bf67-vjrqn\" (UID: \"cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn"
Oct 30 21:47:44.316335 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:44.316138    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d-workload-certs\") pod \"maas-default-gateway-openshift-default-7df854bf67-vjrqn\" (UID: \"cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn"
Oct 30 21:47:44.316335 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:44.316172    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d-workload-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-vjrqn\" (UID: \"cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn"
Oct 30 21:47:44.316335 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:44.316195    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d-credential-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-vjrqn\" (UID: \"cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn"
Oct 30 21:47:44.316544 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:44.316361    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d-istio-token\") pod \"maas-default-gateway-openshift-default-7df854bf67-vjrqn\" (UID: \"cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn"
Oct 30 21:47:44.316544 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:44.316465    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d-istio-data\") pod \"maas-default-gateway-openshift-default-7df854bf67-vjrqn\" (UID: \"cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn"
Oct 30 21:47:44.316656 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:44.316573    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7df854bf67-vjrqn\" (UID: \"cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn"
Oct 30 21:47:44.316656 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:44.316602    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7df854bf67-vjrqn\" (UID: \"cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn"
Oct 30 21:47:44.316656 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:44.316632    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdlqs\" (UniqueName: \"kubernetes.io/projected/cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d-kube-api-access-mdlqs\") pod \"maas-default-gateway-openshift-default-7df854bf67-vjrqn\" (UID: \"cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn"
Oct 30 21:47:44.316806 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:44.316728    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d-workload-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-vjrqn\" (UID: \"cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn"
Oct 30 21:47:44.316879 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:44.316856    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d-workload-certs\") pod \"maas-default-gateway-openshift-default-7df854bf67-vjrqn\" (UID: \"cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn"
Oct 30 21:47:44.316939 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:44.316863    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d-istio-data\") pod \"maas-default-gateway-openshift-default-7df854bf67-vjrqn\" (UID: \"cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn"
Oct 30 21:47:44.317043 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:44.317022    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d-credential-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-vjrqn\" (UID: \"cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn"
Oct 30 21:47:44.317437 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:44.317420    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7df854bf67-vjrqn\" (UID: \"cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn"
Oct 30 21:47:44.318528 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:44.318507    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d-istio-envoy\") pod \"maas-default-gateway-openshift-default-7df854bf67-vjrqn\" (UID: \"cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn"
Oct 30 21:47:44.319454 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:44.319431    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7df854bf67-vjrqn\" (UID: \"cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn"
Oct 30 21:47:44.375862 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:44.375827    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d-istio-token\") pod \"maas-default-gateway-openshift-default-7df854bf67-vjrqn\" (UID: \"cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn"
Oct 30 21:47:44.376055 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:44.376030    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdlqs\" (UniqueName: \"kubernetes.io/projected/cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d-kube-api-access-mdlqs\") pod \"maas-default-gateway-openshift-default-7df854bf67-vjrqn\" (UID: \"cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn"
Oct 30 21:47:44.402558 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:44.402519    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn"
Oct 30 21:47:44.884784 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:44.884755    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn"]
Oct 30 21:47:44.887355 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:47:44.887315    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc9ba2a0_4d99_4e16_96e5_0db93a3ec75d.slice/crio-98f273303171b4a8fe7dbcddec9b89d51f63d649c9b343d20f8be462e2a01b47 WatchSource:0}: Error finding container 98f273303171b4a8fe7dbcddec9b89d51f63d649c9b343d20f8be462e2a01b47: Status 404 returned error can't find the container with id 98f273303171b4a8fe7dbcddec9b89d51f63d649c9b343d20f8be462e2a01b47
Oct 30 21:47:44.889489 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:44.889450    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:47:44.889562 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:44.889526    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:47:44.889562 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:44.889558    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:47:45.874302 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:45.874261    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn" event={"ID":"cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d","Type":"ContainerStarted","Data":"b4d010cf043391bef9d200cefbb06c78545d1b93f3121f12efe148080d351fff"}
Oct 30 21:47:45.874844 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:45.874335    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn" event={"ID":"cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d","Type":"ContainerStarted","Data":"98f273303171b4a8fe7dbcddec9b89d51f63d649c9b343d20f8be462e2a01b47"}
Oct 30 21:47:46.041281 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:46.041209    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn" podStartSLOduration=2.041195165 podStartE2EDuration="2.041195165s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:47:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:47:46.038848259 +0000 UTC m=+608.216336876" watchObservedRunningTime="2026-10-30 21:47:46.041195165 +0000 UTC m=+608.218683782"
Oct 30 21:47:46.419262 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:46.407591    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn"
Oct 30 21:47:46.427324 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:46.424390    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-vjrqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 30 21:47:46.427324 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:46.424465    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn" podUID="cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 30 21:47:47.071976 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:47.071948    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:47.409264 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:47.403126    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-vjrqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 30 21:47:47.409622 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:47.409581    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn" podUID="cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 30 21:47:48.427406 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:48.427366    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-vjrqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 30 21:47:48.427795 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:48.427436    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn" podUID="cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 30 21:47:49.413695 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:49.413651    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-vjrqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 30 21:47:49.414031 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:49.414001    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn" podUID="cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 30 21:47:50.421262 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:50.414718    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-vjrqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 30 21:47:50.421262 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:50.414790    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn" podUID="cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 30 21:47:51.403855 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:51.403810    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-vjrqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 30 21:47:51.404038 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:51.403898    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn" podUID="cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 30 21:47:52.403323 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:52.403282    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-vjrqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 30 21:47:52.403770 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:52.403365    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn" podUID="cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 30 21:47:53.475097 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:53.475055    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn"
Oct 30 21:47:53.475576 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:53.475557    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn"
Oct 30 21:47:53.476480 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:53.476458    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-vjrqn"
Oct 30 21:47:59.638829 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:59.638794    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-64d8b4548c-bhhfv"]
Oct 30 21:47:59.642573 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:59.642551    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-64d8b4548c-bhhfv"
Oct 30 21:47:59.697021 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:59.696991    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-64d8b4548c-bhhfv"]
Oct 30 21:47:59.740730 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:59.740699    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-kd7kz\""
Oct 30 21:47:59.740912 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:59.740806    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Oct 30 21:47:59.766608 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:59.766574    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pv2zh\" (UniqueName: \"kubernetes.io/projected/89d9923c-dc8d-4655-be5d-ed3c9c36d7b2-kube-api-access-pv2zh\") pod \"authorino-64d8b4548c-bhhfv\" (UID: \"89d9923c-dc8d-4655-be5d-ed3c9c36d7b2\") " pod="kuadrant-system/authorino-64d8b4548c-bhhfv"
Oct 30 21:47:59.766608 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:59.766612    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/89d9923c-dc8d-4655-be5d-ed3c9c36d7b2-tls-cert\") pod \"authorino-64d8b4548c-bhhfv\" (UID: \"89d9923c-dc8d-4655-be5d-ed3c9c36d7b2\") " pod="kuadrant-system/authorino-64d8b4548c-bhhfv"
Oct 30 21:47:59.867611 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:59.867570    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv2zh\" (UniqueName: \"kubernetes.io/projected/89d9923c-dc8d-4655-be5d-ed3c9c36d7b2-kube-api-access-pv2zh\") pod \"authorino-64d8b4548c-bhhfv\" (UID: \"89d9923c-dc8d-4655-be5d-ed3c9c36d7b2\") " pod="kuadrant-system/authorino-64d8b4548c-bhhfv"
Oct 30 21:47:59.867611 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:59.867609    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/89d9923c-dc8d-4655-be5d-ed3c9c36d7b2-tls-cert\") pod \"authorino-64d8b4548c-bhhfv\" (UID: \"89d9923c-dc8d-4655-be5d-ed3c9c36d7b2\") " pod="kuadrant-system/authorino-64d8b4548c-bhhfv"
Oct 30 21:47:59.870345 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:59.870310    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/89d9923c-dc8d-4655-be5d-ed3c9c36d7b2-tls-cert\") pod \"authorino-64d8b4548c-bhhfv\" (UID: \"89d9923c-dc8d-4655-be5d-ed3c9c36d7b2\") " pod="kuadrant-system/authorino-64d8b4548c-bhhfv"
Oct 30 21:47:59.927694 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:59.927612    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pv2zh\" (UniqueName: \"kubernetes.io/projected/89d9923c-dc8d-4655-be5d-ed3c9c36d7b2-kube-api-access-pv2zh\") pod \"authorino-64d8b4548c-bhhfv\" (UID: \"89d9923c-dc8d-4655-be5d-ed3c9c36d7b2\") " pod="kuadrant-system/authorino-64d8b4548c-bhhfv"
Oct 30 21:47:59.955197 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:47:59.955167    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-64d8b4548c-bhhfv"
Oct 30 21:48:00.328232 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:48:00.328193    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-64d8b4548c-bhhfv"]
Oct 30 21:48:00.330008 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:48:00.329964    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod89d9923c_dc8d_4655_be5d_ed3c9c36d7b2.slice/crio-6c3955ec5a80adad9aa89271810df4b5f545d4311badbb0786fd1bbbda1aef67 WatchSource:0}: Error finding container 6c3955ec5a80adad9aa89271810df4b5f545d4311badbb0786fd1bbbda1aef67: Status 404 returned error can't find the container with id 6c3955ec5a80adad9aa89271810df4b5f545d4311badbb0786fd1bbbda1aef67
Oct 30 21:48:00.403617 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:48:00.403586    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:00.637484 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:48:00.637439    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:00.977163 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:48:00.977064    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-64d8b4548c-bhhfv" event={"ID":"89d9923c-dc8d-4655-be5d-ed3c9c36d7b2","Type":"ContainerStarted","Data":"6c3955ec5a80adad9aa89271810df4b5f545d4311badbb0786fd1bbbda1aef67"}
Oct 30 21:48:02.183669 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:48:02.183633    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:02.396459 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:48:02.396414    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:02.990074 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:48:02.990035    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-64d8b4548c-bhhfv" event={"ID":"89d9923c-dc8d-4655-be5d-ed3c9c36d7b2","Type":"ContainerStarted","Data":"14bdae5f684d74018d66847058c24bb189cf476b135c0817b8309590ac1a0371"}
Oct 30 21:48:03.115215 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:48:03.115162    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-64d8b4548c-bhhfv" podStartSLOduration=1.880030141 podStartE2EDuration="4.115147745s" totalImagesPullingTime="2.235117604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:47:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:48:03.112509735 +0000 UTC m=+625.289998344" watchObservedRunningTime="2026-10-30 21:48:03.115147745 +0000 UTC m=+625.292636387"
Oct 30 21:48:05.570170 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:48:05.570134    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:48:28.424787 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:48:28.424736    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:28.586643 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:48:28.586599    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:31.384479 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:48:31.384447    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:34.944583 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:48:34.944544    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:35.107455 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:48:35.107416    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:35.568181 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:48:35.568144    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:49:01.475151 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:01.475050    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:01.671920 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:01.671882    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:01.854144 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:01.854037    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:02.045661 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.045629    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:02.168876 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.168841    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"]
Oct 30 21:49:02.173033 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.172991    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"
Oct 30 21:49:02.227417 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.227387    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-a1d107b0-openshift-default-dockercfg-79rmn\""
Oct 30 21:49:02.265994 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.265960    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d11e9475-1d27-44b3-b433-13a5cc49a13a-credential-socket\") pod \"e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw\" (UID: \"d11e9475-1d27-44b3-b433-13a5cc49a13a\") " pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"
Oct 30 21:49:02.266225 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.266022    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d11e9475-1d27-44b3-b433-13a5cc49a13a-istio-data\") pod \"e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw\" (UID: \"d11e9475-1d27-44b3-b433-13a5cc49a13a\") " pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"
Oct 30 21:49:02.266225 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.266044    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d11e9475-1d27-44b3-b433-13a5cc49a13a-istio-envoy\") pod \"e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw\" (UID: \"d11e9475-1d27-44b3-b433-13a5cc49a13a\") " pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"
Oct 30 21:49:02.266225 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.266190    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d11e9475-1d27-44b3-b433-13a5cc49a13a-workload-socket\") pod \"e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw\" (UID: \"d11e9475-1d27-44b3-b433-13a5cc49a13a\") " pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"
Oct 30 21:49:02.266350 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.266214    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d11e9475-1d27-44b3-b433-13a5cc49a13a-istio-podinfo\") pod \"e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw\" (UID: \"d11e9475-1d27-44b3-b433-13a5cc49a13a\") " pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"
Oct 30 21:49:02.266350 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.266267    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d11e9475-1d27-44b3-b433-13a5cc49a13a-istiod-ca-cert\") pod \"e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw\" (UID: \"d11e9475-1d27-44b3-b433-13a5cc49a13a\") " pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"
Oct 30 21:49:02.266415 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.266343    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8gjm\" (UniqueName: \"kubernetes.io/projected/d11e9475-1d27-44b3-b433-13a5cc49a13a-kube-api-access-z8gjm\") pod \"e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw\" (UID: \"d11e9475-1d27-44b3-b433-13a5cc49a13a\") " pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"
Oct 30 21:49:02.266415 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.266370    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d11e9475-1d27-44b3-b433-13a5cc49a13a-istio-token\") pod \"e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw\" (UID: \"d11e9475-1d27-44b3-b433-13a5cc49a13a\") " pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"
Oct 30 21:49:02.266481 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.266416    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d11e9475-1d27-44b3-b433-13a5cc49a13a-workload-certs\") pod \"e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw\" (UID: \"d11e9475-1d27-44b3-b433-13a5cc49a13a\") " pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"
Oct 30 21:49:02.274934 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.274895    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"]
Oct 30 21:49:02.297070 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.297034    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:02.367690 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.367646    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d11e9475-1d27-44b3-b433-13a5cc49a13a-workload-socket\") pod \"e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw\" (UID: \"d11e9475-1d27-44b3-b433-13a5cc49a13a\") " pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"
Oct 30 21:49:02.367690 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.367688    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d11e9475-1d27-44b3-b433-13a5cc49a13a-istio-podinfo\") pod \"e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw\" (UID: \"d11e9475-1d27-44b3-b433-13a5cc49a13a\") " pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"
Oct 30 21:49:02.368017 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.367834    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d11e9475-1d27-44b3-b433-13a5cc49a13a-istiod-ca-cert\") pod \"e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw\" (UID: \"d11e9475-1d27-44b3-b433-13a5cc49a13a\") " pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"
Oct 30 21:49:02.368017 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.367898    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8gjm\" (UniqueName: \"kubernetes.io/projected/d11e9475-1d27-44b3-b433-13a5cc49a13a-kube-api-access-z8gjm\") pod \"e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw\" (UID: \"d11e9475-1d27-44b3-b433-13a5cc49a13a\") " pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"
Oct 30 21:49:02.368017 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.367926    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d11e9475-1d27-44b3-b433-13a5cc49a13a-istio-token\") pod \"e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw\" (UID: \"d11e9475-1d27-44b3-b433-13a5cc49a13a\") " pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"
Oct 30 21:49:02.368017 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.368002    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d11e9475-1d27-44b3-b433-13a5cc49a13a-workload-certs\") pod \"e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw\" (UID: \"d11e9475-1d27-44b3-b433-13a5cc49a13a\") " pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"
Oct 30 21:49:02.368222 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.368067    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d11e9475-1d27-44b3-b433-13a5cc49a13a-credential-socket\") pod \"e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw\" (UID: \"d11e9475-1d27-44b3-b433-13a5cc49a13a\") " pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"
Oct 30 21:49:02.368222 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.368133    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d11e9475-1d27-44b3-b433-13a5cc49a13a-istio-data\") pod \"e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw\" (UID: \"d11e9475-1d27-44b3-b433-13a5cc49a13a\") " pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"
Oct 30 21:49:02.368222 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.368152    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d11e9475-1d27-44b3-b433-13a5cc49a13a-workload-socket\") pod \"e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw\" (UID: \"d11e9475-1d27-44b3-b433-13a5cc49a13a\") " pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"
Oct 30 21:49:02.368455 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.368386    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d11e9475-1d27-44b3-b433-13a5cc49a13a-workload-certs\") pod \"e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw\" (UID: \"d11e9475-1d27-44b3-b433-13a5cc49a13a\") " pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"
Oct 30 21:49:02.368528 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.368449    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d11e9475-1d27-44b3-b433-13a5cc49a13a-istio-data\") pod \"e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw\" (UID: \"d11e9475-1d27-44b3-b433-13a5cc49a13a\") " pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"
Oct 30 21:49:02.368528 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.368487    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d11e9475-1d27-44b3-b433-13a5cc49a13a-credential-socket\") pod \"e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw\" (UID: \"d11e9475-1d27-44b3-b433-13a5cc49a13a\") " pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"
Oct 30 21:49:02.368528 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.368507    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d11e9475-1d27-44b3-b433-13a5cc49a13a-istio-envoy\") pod \"e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw\" (UID: \"d11e9475-1d27-44b3-b433-13a5cc49a13a\") " pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"
Oct 30 21:49:02.368660 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.368647    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d11e9475-1d27-44b3-b433-13a5cc49a13a-istiod-ca-cert\") pod \"e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw\" (UID: \"d11e9475-1d27-44b3-b433-13a5cc49a13a\") " pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"
Oct 30 21:49:02.370232 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.370208    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d11e9475-1d27-44b3-b433-13a5cc49a13a-istio-envoy\") pod \"e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw\" (UID: \"d11e9475-1d27-44b3-b433-13a5cc49a13a\") " pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"
Oct 30 21:49:02.370347 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.370281    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d11e9475-1d27-44b3-b433-13a5cc49a13a-istio-podinfo\") pod \"e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw\" (UID: \"d11e9475-1d27-44b3-b433-13a5cc49a13a\") " pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"
Oct 30 21:49:02.428627 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.428557    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d11e9475-1d27-44b3-b433-13a5cc49a13a-istio-token\") pod \"e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw\" (UID: \"d11e9475-1d27-44b3-b433-13a5cc49a13a\") " pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"
Oct 30 21:49:02.428627 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.428591    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8gjm\" (UniqueName: \"kubernetes.io/projected/d11e9475-1d27-44b3-b433-13a5cc49a13a-kube-api-access-z8gjm\") pod \"e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw\" (UID: \"d11e9475-1d27-44b3-b433-13a5cc49a13a\") " pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"
Oct 30 21:49:02.493052 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.493016    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"
Oct 30 21:49:02.495460 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.495432    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:02.675867 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.675803    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:02.901631 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.901594    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:02.942024 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.941994    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"]
Oct 30 21:49:02.943749 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:49:02.943719    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd11e9475_1d27_44b3_b433_13a5cc49a13a.slice/crio-e32a7b1eee2788a161e819c40baa44044b9bcfb28da18e62e41f6d73accfd953 WatchSource:0}: Error finding container e32a7b1eee2788a161e819c40baa44044b9bcfb28da18e62e41f6d73accfd953: Status 404 returned error can't find the container with id e32a7b1eee2788a161e819c40baa44044b9bcfb28da18e62e41f6d73accfd953
Oct 30 21:49:02.945950 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.945917    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:49:02.946051 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.945986    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:49:02.946051 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:02.946012    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:49:03.163390 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.163294    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"]
Oct 30 21:49:03.166951 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.166920    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"
Oct 30 21:49:03.216755 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.216725    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-16c49820-gw-openshift-default-dockercfg-2rj2h\""
Oct 30 21:49:03.227852 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.227011    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"]
Oct 30 21:49:03.278155 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.278120    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e626a30e-ff4e-4e87-9f50-000865cfe6c0-istio-token\") pod \"e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn\" (UID: \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\") " pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"
Oct 30 21:49:03.278155 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.278160    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e626a30e-ff4e-4e87-9f50-000865cfe6c0-workload-certs\") pod \"e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn\" (UID: \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\") " pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"
Oct 30 21:49:03.278389 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.278304    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e626a30e-ff4e-4e87-9f50-000865cfe6c0-credential-socket\") pod \"e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn\" (UID: \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\") " pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"
Oct 30 21:49:03.278468 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.278451    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e626a30e-ff4e-4e87-9f50-000865cfe6c0-istio-envoy\") pod \"e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn\" (UID: \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\") " pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"
Oct 30 21:49:03.278504 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.278492    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e626a30e-ff4e-4e87-9f50-000865cfe6c0-istio-data\") pod \"e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn\" (UID: \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\") " pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"
Oct 30 21:49:03.278628 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.278610    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e626a30e-ff4e-4e87-9f50-000865cfe6c0-istio-podinfo\") pod \"e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn\" (UID: \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\") " pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"
Oct 30 21:49:03.278677 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.278635    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbkmg\" (UniqueName: \"kubernetes.io/projected/e626a30e-ff4e-4e87-9f50-000865cfe6c0-kube-api-access-mbkmg\") pod \"e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn\" (UID: \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\") " pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"
Oct 30 21:49:03.278677 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.278657    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e626a30e-ff4e-4e87-9f50-000865cfe6c0-workload-socket\") pod \"e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn\" (UID: \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\") " pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"
Oct 30 21:49:03.278738 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.278680    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e626a30e-ff4e-4e87-9f50-000865cfe6c0-istiod-ca-cert\") pod \"e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn\" (UID: \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\") " pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"
Oct 30 21:49:03.286099 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.286067    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:03.338903 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.338860    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw" event={"ID":"d11e9475-1d27-44b3-b433-13a5cc49a13a","Type":"ContainerStarted","Data":"948826c218be85302c71f09ea7e45d8636ac493c23c162e3f1d7c4a1cfcf6c75"}
Oct 30 21:49:03.339159 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.338933    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw" event={"ID":"d11e9475-1d27-44b3-b433-13a5cc49a13a","Type":"ContainerStarted","Data":"e32a7b1eee2788a161e819c40baa44044b9bcfb28da18e62e41f6d73accfd953"}
Oct 30 21:49:03.379566 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.379525    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e626a30e-ff4e-4e87-9f50-000865cfe6c0-istio-token\") pod \"e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn\" (UID: \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\") " pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"
Oct 30 21:49:03.379762 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.379573    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e626a30e-ff4e-4e87-9f50-000865cfe6c0-workload-certs\") pod \"e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn\" (UID: \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\") " pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"
Oct 30 21:49:03.379762 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.379608    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e626a30e-ff4e-4e87-9f50-000865cfe6c0-credential-socket\") pod \"e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn\" (UID: \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\") " pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"
Oct 30 21:49:03.379762 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.379638    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e626a30e-ff4e-4e87-9f50-000865cfe6c0-istio-envoy\") pod \"e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn\" (UID: \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\") " pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"
Oct 30 21:49:03.379762 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.379678    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e626a30e-ff4e-4e87-9f50-000865cfe6c0-istio-data\") pod \"e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn\" (UID: \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\") " pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"
Oct 30 21:49:03.379762 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.379731    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e626a30e-ff4e-4e87-9f50-000865cfe6c0-istio-podinfo\") pod \"e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn\" (UID: \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\") " pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"
Oct 30 21:49:03.379762 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.379755    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbkmg\" (UniqueName: \"kubernetes.io/projected/e626a30e-ff4e-4e87-9f50-000865cfe6c0-kube-api-access-mbkmg\") pod \"e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn\" (UID: \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\") " pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"
Oct 30 21:49:03.380079 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.379791    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e626a30e-ff4e-4e87-9f50-000865cfe6c0-workload-socket\") pod \"e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn\" (UID: \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\") " pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"
Oct 30 21:49:03.380079 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.379830    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e626a30e-ff4e-4e87-9f50-000865cfe6c0-istiod-ca-cert\") pod \"e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn\" (UID: \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\") " pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"
Oct 30 21:49:03.380590 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.380564    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e626a30e-ff4e-4e87-9f50-000865cfe6c0-workload-certs\") pod \"e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn\" (UID: \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\") " pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"
Oct 30 21:49:03.380750 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.380626    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e626a30e-ff4e-4e87-9f50-000865cfe6c0-credential-socket\") pod \"e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn\" (UID: \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\") " pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"
Oct 30 21:49:03.380854 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.380652    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e626a30e-ff4e-4e87-9f50-000865cfe6c0-istio-data\") pod \"e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn\" (UID: \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\") " pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"
Oct 30 21:49:03.380854 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.380728    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e626a30e-ff4e-4e87-9f50-000865cfe6c0-workload-socket\") pod \"e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn\" (UID: \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\") " pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"
Oct 30 21:49:03.381052 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.381029    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e626a30e-ff4e-4e87-9f50-000865cfe6c0-istiod-ca-cert\") pod \"e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn\" (UID: \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\") " pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"
Oct 30 21:49:03.395668 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.382944    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e626a30e-ff4e-4e87-9f50-000865cfe6c0-istio-envoy\") pod \"e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn\" (UID: \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\") " pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"
Oct 30 21:49:03.395668 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.383074    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e626a30e-ff4e-4e87-9f50-000865cfe6c0-istio-podinfo\") pod \"e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn\" (UID: \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\") " pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"
Oct 30 21:49:03.438972 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.438896    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e626a30e-ff4e-4e87-9f50-000865cfe6c0-istio-token\") pod \"e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn\" (UID: \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\") " pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"
Oct 30 21:49:03.439199 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.439174    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbkmg\" (UniqueName: \"kubernetes.io/projected/e626a30e-ff4e-4e87-9f50-000865cfe6c0-kube-api-access-mbkmg\") pod \"e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn\" (UID: \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\") " pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"
Oct 30 21:49:03.459208 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.459176    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:03.466730 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.466672    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw" podStartSLOduration=1.46665826 podStartE2EDuration="1.46665826s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:03.464782695 +0000 UTC m=+685.642271327" watchObservedRunningTime="2026-10-30 21:49:03.46665826 +0000 UTC m=+685.644146877"
Oct 30 21:49:03.480885 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.480829    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"
Oct 30 21:49:03.494140 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.494116    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"
Oct 30 21:49:03.500225 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.500191    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"
Oct 30 21:49:03.620648 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.620606    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:03.873642 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.873605    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"]
Oct 30 21:49:03.875131 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:49:03.875090    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode626a30e_ff4e_4e87_9f50_000865cfe6c0.slice/crio-71f1d5e2e65fe2f9f6bae5449c806da2209d6391d8cedb4a66d7fc4a89cc7d39 WatchSource:0}: Error finding container 71f1d5e2e65fe2f9f6bae5449c806da2209d6391d8cedb4a66d7fc4a89cc7d39: Status 404 returned error can't find the container with id 71f1d5e2e65fe2f9f6bae5449c806da2209d6391d8cedb4a66d7fc4a89cc7d39
Oct 30 21:49:03.878155 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.878120    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:49:03.878311 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.878192    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:49:03.878311 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:03.878224    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:49:04.347546 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:04.347437    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn" event={"ID":"e626a30e-ff4e-4e87-9f50-000865cfe6c0","Type":"ContainerStarted","Data":"94fa926fb003c56e36a68c5a8168d7eafa9f6658ce73dcbfd25763738d50376b"}
Oct 30 21:49:04.347546 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:04.347504    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn" event={"ID":"e626a30e-ff4e-4e87-9f50-000865cfe6c0","Type":"ContainerStarted","Data":"71f1d5e2e65fe2f9f6bae5449c806da2209d6391d8cedb4a66d7fc4a89cc7d39"}
Oct 30 21:49:04.347808 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:04.347785    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"
Oct 30 21:49:04.348789 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:04.348765    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"
Oct 30 21:49:04.466678 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:04.466601    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn" podStartSLOduration=1.466582824 podStartE2EDuration="1.466582824s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:04.466560575 +0000 UTC m=+686.644049193" watchObservedRunningTime="2026-10-30 21:49:04.466582824 +0000 UTC m=+686.644071442"
Oct 30 21:49:04.481876 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:04.481842    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"
Oct 30 21:49:04.486907 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:04.486877    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"
Oct 30 21:49:04.890350 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:04.890304    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:05.116599 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:05.116560    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:05.354427 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:05.354332    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"
Oct 30 21:49:05.355211 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:05.355193    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"
Oct 30 21:49:05.514936 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:05.514902    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:05.568371 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:49:05.568334    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:49:05.684580 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:05.684544    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:05.845540 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:05.845505    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:06.283575 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:06.283543    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:06.479301 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:06.479267    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:06.683027 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:06.682994    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:08.687058 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:08.687019    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:08.848326 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:08.848288    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:09.159657 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:09.159616    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:09.551985 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:09.551890    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"]
Oct 30 21:49:09.556813 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:09.556787    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"
Oct 30 21:49:09.607666 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:09.607624    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-ffe7a1cf-openshift-default-dockercfg-g28kp\""
Oct 30 21:49:09.655578 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:09.655544    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03d16436-58d3-495e-84d4-38b6d3b55a8b-istio-envoy\") pod \"e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg\" (UID: \"03d16436-58d3-495e-84d4-38b6d3b55a8b\") " pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"
Oct 30 21:49:09.655578 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:09.655580    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03d16436-58d3-495e-84d4-38b6d3b55a8b-istio-data\") pod \"e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg\" (UID: \"03d16436-58d3-495e-84d4-38b6d3b55a8b\") " pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"
Oct 30 21:49:09.655879 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:09.655602    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03d16436-58d3-495e-84d4-38b6d3b55a8b-workload-socket\") pod \"e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg\" (UID: \"03d16436-58d3-495e-84d4-38b6d3b55a8b\") " pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"
Oct 30 21:49:09.655879 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:09.655780    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03d16436-58d3-495e-84d4-38b6d3b55a8b-workload-certs\") pod \"e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg\" (UID: \"03d16436-58d3-495e-84d4-38b6d3b55a8b\") " pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"
Oct 30 21:49:09.655879 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:09.655838    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03d16436-58d3-495e-84d4-38b6d3b55a8b-istio-podinfo\") pod \"e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg\" (UID: \"03d16436-58d3-495e-84d4-38b6d3b55a8b\") " pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"
Oct 30 21:49:09.656017 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:09.655894    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03d16436-58d3-495e-84d4-38b6d3b55a8b-istio-token\") pod \"e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg\" (UID: \"03d16436-58d3-495e-84d4-38b6d3b55a8b\") " pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"
Oct 30 21:49:09.656017 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:09.655985    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03d16436-58d3-495e-84d4-38b6d3b55a8b-credential-socket\") pod \"e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg\" (UID: \"03d16436-58d3-495e-84d4-38b6d3b55a8b\") " pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"
Oct 30 21:49:09.656098 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:09.656028    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2vdh\" (UniqueName: \"kubernetes.io/projected/03d16436-58d3-495e-84d4-38b6d3b55a8b-kube-api-access-v2vdh\") pod \"e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg\" (UID: \"03d16436-58d3-495e-84d4-38b6d3b55a8b\") " pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"
Oct 30 21:49:09.656098 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:09.656066    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03d16436-58d3-495e-84d4-38b6d3b55a8b-istiod-ca-cert\") pod \"e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg\" (UID: \"03d16436-58d3-495e-84d4-38b6d3b55a8b\") " pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"
Oct 30 21:49:09.656589 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:09.656571    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"]
Oct 30 21:49:09.661741 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:09.661718    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:09.757417 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:09.757365    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03d16436-58d3-495e-84d4-38b6d3b55a8b-credential-socket\") pod \"e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg\" (UID: \"03d16436-58d3-495e-84d4-38b6d3b55a8b\") " pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"
Oct 30 21:49:09.757417 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:09.757422    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2vdh\" (UniqueName: \"kubernetes.io/projected/03d16436-58d3-495e-84d4-38b6d3b55a8b-kube-api-access-v2vdh\") pod \"e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg\" (UID: \"03d16436-58d3-495e-84d4-38b6d3b55a8b\") " pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"
Oct 30 21:49:09.757977 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:09.757469    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03d16436-58d3-495e-84d4-38b6d3b55a8b-istiod-ca-cert\") pod \"e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg\" (UID: \"03d16436-58d3-495e-84d4-38b6d3b55a8b\") " pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"
Oct 30 21:49:09.757977 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:09.757499    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03d16436-58d3-495e-84d4-38b6d3b55a8b-istio-envoy\") pod \"e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg\" (UID: \"03d16436-58d3-495e-84d4-38b6d3b55a8b\") " pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"
Oct 30 21:49:09.757977 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:09.757515    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03d16436-58d3-495e-84d4-38b6d3b55a8b-istio-data\") pod \"e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg\" (UID: \"03d16436-58d3-495e-84d4-38b6d3b55a8b\") " pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"
Oct 30 21:49:09.757977 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:09.757536    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03d16436-58d3-495e-84d4-38b6d3b55a8b-workload-socket\") pod \"e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg\" (UID: \"03d16436-58d3-495e-84d4-38b6d3b55a8b\") " pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"
Oct 30 21:49:09.757977 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:09.757593    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03d16436-58d3-495e-84d4-38b6d3b55a8b-workload-certs\") pod \"e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg\" (UID: \"03d16436-58d3-495e-84d4-38b6d3b55a8b\") " pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"
Oct 30 21:49:09.757977 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:09.757651    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03d16436-58d3-495e-84d4-38b6d3b55a8b-istio-podinfo\") pod \"e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg\" (UID: \"03d16436-58d3-495e-84d4-38b6d3b55a8b\") " pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"
Oct 30 21:49:09.757977 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:09.757677    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03d16436-58d3-495e-84d4-38b6d3b55a8b-istio-token\") pod \"e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg\" (UID: \"03d16436-58d3-495e-84d4-38b6d3b55a8b\") " pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"
Oct 30 21:49:09.757977 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:09.757868    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03d16436-58d3-495e-84d4-38b6d3b55a8b-credential-socket\") pod \"e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg\" (UID: \"03d16436-58d3-495e-84d4-38b6d3b55a8b\") " pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"
Oct 30 21:49:09.757977 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:09.757942    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03d16436-58d3-495e-84d4-38b6d3b55a8b-workload-socket\") pod \"e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg\" (UID: \"03d16436-58d3-495e-84d4-38b6d3b55a8b\") " pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"
Oct 30 21:49:09.758379 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:09.758070    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03d16436-58d3-495e-84d4-38b6d3b55a8b-istio-data\") pod \"e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg\" (UID: \"03d16436-58d3-495e-84d4-38b6d3b55a8b\") " pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"
Oct 30 21:49:09.758379 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:09.758105    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03d16436-58d3-495e-84d4-38b6d3b55a8b-workload-certs\") pod \"e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg\" (UID: \"03d16436-58d3-495e-84d4-38b6d3b55a8b\") " pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"
Oct 30 21:49:09.758447 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:09.758383    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03d16436-58d3-495e-84d4-38b6d3b55a8b-istiod-ca-cert\") pod \"e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg\" (UID: \"03d16436-58d3-495e-84d4-38b6d3b55a8b\") " pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"
Oct 30 21:49:09.760014 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:09.759990    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03d16436-58d3-495e-84d4-38b6d3b55a8b-istio-envoy\") pod \"e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg\" (UID: \"03d16436-58d3-495e-84d4-38b6d3b55a8b\") " pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"
Oct 30 21:49:09.760199 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:09.760181    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03d16436-58d3-495e-84d4-38b6d3b55a8b-istio-podinfo\") pod \"e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg\" (UID: \"03d16436-58d3-495e-84d4-38b6d3b55a8b\") " pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"
Oct 30 21:49:09.816268 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:09.816181    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2vdh\" (UniqueName: \"kubernetes.io/projected/03d16436-58d3-495e-84d4-38b6d3b55a8b-kube-api-access-v2vdh\") pod \"e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg\" (UID: \"03d16436-58d3-495e-84d4-38b6d3b55a8b\") " pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"
Oct 30 21:49:09.816268 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:09.816183    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03d16436-58d3-495e-84d4-38b6d3b55a8b-istio-token\") pod \"e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg\" (UID: \"03d16436-58d3-495e-84d4-38b6d3b55a8b\") " pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"
Oct 30 21:49:09.870901 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:09.870864    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"
Oct 30 21:49:09.881761 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:09.881728    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:10.137458 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:10.137421    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:10.301480 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:10.301451    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"]
Oct 30 21:49:10.303427 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:49:10.303398    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03d16436_58d3_495e_84d4_38b6d3b55a8b.slice/crio-4cae6cc89ccddea92bad4a3e2feca061190f93b141d2d8cca915986cd82476ae WatchSource:0}: Error finding container 4cae6cc89ccddea92bad4a3e2feca061190f93b141d2d8cca915986cd82476ae: Status 404 returned error can't find the container with id 4cae6cc89ccddea92bad4a3e2feca061190f93b141d2d8cca915986cd82476ae
Oct 30 21:49:10.305724 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:10.305693    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:49:10.305802 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:10.305755    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:49:10.305802 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:10.305786    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:49:10.331362 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:10.331326    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:10.408300 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:10.408217    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg" event={"ID":"03d16436-58d3-495e-84d4-38b6d3b55a8b","Type":"ContainerStarted","Data":"4cae6cc89ccddea92bad4a3e2feca061190f93b141d2d8cca915986cd82476ae"}
Oct 30 21:49:10.538519 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:10.538479    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:10.720907 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:10.720823    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:10.928174 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:10.928108    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:11.417956 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:11.417913    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg" event={"ID":"03d16436-58d3-495e-84d4-38b6d3b55a8b","Type":"ContainerStarted","Data":"ded087dbe4633a08e6dbfe18cf8f74717787fd47a7044fda7a3cb81004168ee5"}
Oct 30 21:49:11.582166 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:11.581132    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg" podStartSLOduration=2.5811132949999998 podStartE2EDuration="2.581113295s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:11.579337888 +0000 UTC m=+693.756826504" watchObservedRunningTime="2026-10-30 21:49:11.581113295 +0000 UTC m=+693.758601909"
Oct 30 21:49:11.788760 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:11.788670    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:11.871323 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:11.871287    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"
Oct 30 21:49:11.876852 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:11.876821    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"
Oct 30 21:49:11.942844 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:11.942808    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:12.099701 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:12.099609    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:12.423548 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:12.423509    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"
Oct 30 21:49:12.424457 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:12.424436    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"
Oct 30 21:49:13.842376 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:13.842342    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:14.017071 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:14.017033    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:14.246641 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:14.246603    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ebf644d4-6b5d7b4747-7jqqd"]
Oct 30 21:49:14.250418 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:14.250396    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ebf644d4-6b5d7b4747-7jqqd"
Oct 30 21:49:14.300993 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:14.300957    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/452e61af-d766-4929-b607-a0ad2332b7cf-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ebf644d4-6b5d7b4747-7jqqd\" (UID: \"452e61af-d766-4929-b607-a0ad2332b7cf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ebf644d4-6b5d7b4747-7jqqd"
Oct 30 21:49:14.301163 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:14.301003    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55fc7\" (UniqueName: \"kubernetes.io/projected/452e61af-d766-4929-b607-a0ad2332b7cf-kube-api-access-55fc7\") pod \"payload-pre-processing-e2e-ait-ebf644d4-6b5d7b4747-7jqqd\" (UID: \"452e61af-d766-4929-b607-a0ad2332b7cf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ebf644d4-6b5d7b4747-7jqqd"
Oct 30 21:49:14.305507 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:14.305466    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ebf644d4-6b5d7b4747-7jqqd"]
Oct 30 21:49:14.349942 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:14.349802    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-ebf644d4-dockercfg-6pgct\""
Oct 30 21:49:14.350345 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:14.349863    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-ebf644d4\""
Oct 30 21:49:14.402202 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:14.402057    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/452e61af-d766-4929-b607-a0ad2332b7cf-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ebf644d4-6b5d7b4747-7jqqd\" (UID: \"452e61af-d766-4929-b607-a0ad2332b7cf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ebf644d4-6b5d7b4747-7jqqd"
Oct 30 21:49:14.402202 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:14.402125    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55fc7\" (UniqueName: \"kubernetes.io/projected/452e61af-d766-4929-b607-a0ad2332b7cf-kube-api-access-55fc7\") pod \"payload-pre-processing-e2e-ait-ebf644d4-6b5d7b4747-7jqqd\" (UID: \"452e61af-d766-4929-b607-a0ad2332b7cf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ebf644d4-6b5d7b4747-7jqqd"
Oct 30 21:49:14.403437 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:14.403408    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/452e61af-d766-4929-b607-a0ad2332b7cf-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ebf644d4-6b5d7b4747-7jqqd\" (UID: \"452e61af-d766-4929-b607-a0ad2332b7cf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ebf644d4-6b5d7b4747-7jqqd"
Oct 30 21:49:14.460468 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:14.460438    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55fc7\" (UniqueName: \"kubernetes.io/projected/452e61af-d766-4929-b607-a0ad2332b7cf-kube-api-access-55fc7\") pod \"payload-pre-processing-e2e-ait-ebf644d4-6b5d7b4747-7jqqd\" (UID: \"452e61af-d766-4929-b607-a0ad2332b7cf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ebf644d4-6b5d7b4747-7jqqd"
Oct 30 21:49:14.563818 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:14.563732    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ebf644d4-6b5d7b4747-7jqqd"
Oct 30 21:49:14.951543 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:14.951485    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ebf644d4-6b5d7b4747-7jqqd"]
Oct 30 21:49:14.953010 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:49:14.952977    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod452e61af_d766_4929_b607_a0ad2332b7cf.slice/crio-72979efb8acd30f55f642fac9b8b840487d81b7913aea47f56a00e4ba4c2fcf1 WatchSource:0}: Error finding container 72979efb8acd30f55f642fac9b8b840487d81b7913aea47f56a00e4ba4c2fcf1: Status 404 returned error can't find the container with id 72979efb8acd30f55f642fac9b8b840487d81b7913aea47f56a00e4ba4c2fcf1
Oct 30 21:49:14.955553 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:14.955535    4871 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:49:15.442180 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:15.442141    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ebf644d4-6b5d7b4747-7jqqd" event={"ID":"452e61af-d766-4929-b607-a0ad2332b7cf","Type":"ContainerStarted","Data":"72979efb8acd30f55f642fac9b8b840487d81b7913aea47f56a00e4ba4c2fcf1"}
Oct 30 21:49:16.151145 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:16.151074    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:16.336878 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:16.336843    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:16.495751 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:16.495671    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:16.654800 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:16.654757    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:16.821305 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:16.821187    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-09d8fb9b-54956d9467-lncnc"]
Oct 30 21:49:16.825527 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:16.825493    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-09d8fb9b-54956d9467-lncnc"
Oct 30 21:49:16.827025 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:16.826993    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57ee4e81-90ce-4670-8522-724918f6b037-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-09d8fb9b-54956d9467-lncnc\" (UID: \"57ee4e81-90ce-4670-8522-724918f6b037\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-09d8fb9b-54956d9467-lncnc"
Oct 30 21:49:16.827176 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:16.827065    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29m8k\" (UniqueName: \"kubernetes.io/projected/57ee4e81-90ce-4670-8522-724918f6b037-kube-api-access-29m8k\") pod \"payload-pre-processing-e2e-ait-09d8fb9b-54956d9467-lncnc\" (UID: \"57ee4e81-90ce-4670-8522-724918f6b037\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-09d8fb9b-54956d9467-lncnc"
Oct 30 21:49:16.864622 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:16.864586    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:16.879735 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:16.879700    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-09d8fb9b-54956d9467-lncnc"]
Oct 30 21:49:16.922538 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:16.922489    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-09d8fb9b\""
Oct 30 21:49:16.922538 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:16.922536    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-09d8fb9b-dockercfg-r6nnn\""
Oct 30 21:49:16.928278 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:16.928232    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57ee4e81-90ce-4670-8522-724918f6b037-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-09d8fb9b-54956d9467-lncnc\" (UID: \"57ee4e81-90ce-4670-8522-724918f6b037\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-09d8fb9b-54956d9467-lncnc"
Oct 30 21:49:16.928441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:16.928324    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29m8k\" (UniqueName: \"kubernetes.io/projected/57ee4e81-90ce-4670-8522-724918f6b037-kube-api-access-29m8k\") pod \"payload-pre-processing-e2e-ait-09d8fb9b-54956d9467-lncnc\" (UID: \"57ee4e81-90ce-4670-8522-724918f6b037\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-09d8fb9b-54956d9467-lncnc"
Oct 30 21:49:16.929297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:16.929270    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57ee4e81-90ce-4670-8522-724918f6b037-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-09d8fb9b-54956d9467-lncnc\" (UID: \"57ee4e81-90ce-4670-8522-724918f6b037\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-09d8fb9b-54956d9467-lncnc"
Oct 30 21:49:16.972810 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:16.972775    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-09d8fb9b-7bc65f5f7f-hl4mj"]
Oct 30 21:49:16.977010 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:16.976978    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-09d8fb9b-7bc65f5f7f-hl4mj"
Oct 30 21:49:16.987371 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:16.987345    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-29m8k\" (UniqueName: \"kubernetes.io/projected/57ee4e81-90ce-4670-8522-724918f6b037-kube-api-access-29m8k\") pod \"payload-pre-processing-e2e-ait-09d8fb9b-54956d9467-lncnc\" (UID: \"57ee4e81-90ce-4670-8522-724918f6b037\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-09d8fb9b-54956d9467-lncnc"
Oct 30 21:49:17.013993 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:17.013961    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-09d8fb9b-7bc65f5f7f-hl4mj"]
Oct 30 21:49:17.025663 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:17.025628    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:17.029419 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:17.029390    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t4h9\" (UniqueName: \"kubernetes.io/projected/34f376f3-c14a-4736-9ce5-770d628e800e-kube-api-access-5t4h9\") pod \"payload-processing-e2e-ait-09d8fb9b-7bc65f5f7f-hl4mj\" (UID: \"34f376f3-c14a-4736-9ce5-770d628e800e\") " pod="openshift-ingress/payload-processing-e2e-ait-09d8fb9b-7bc65f5f7f-hl4mj"
Oct 30 21:49:17.030207 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:17.029555    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34f376f3-c14a-4736-9ce5-770d628e800e-plugins-config-volume\") pod \"payload-processing-e2e-ait-09d8fb9b-7bc65f5f7f-hl4mj\" (UID: \"34f376f3-c14a-4736-9ce5-770d628e800e\") " pod="openshift-ingress/payload-processing-e2e-ait-09d8fb9b-7bc65f5f7f-hl4mj"
Oct 30 21:49:17.130215 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:17.130164    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t4h9\" (UniqueName: \"kubernetes.io/projected/34f376f3-c14a-4736-9ce5-770d628e800e-kube-api-access-5t4h9\") pod \"payload-processing-e2e-ait-09d8fb9b-7bc65f5f7f-hl4mj\" (UID: \"34f376f3-c14a-4736-9ce5-770d628e800e\") " pod="openshift-ingress/payload-processing-e2e-ait-09d8fb9b-7bc65f5f7f-hl4mj"
Oct 30 21:49:17.130420 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:17.130311    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34f376f3-c14a-4736-9ce5-770d628e800e-plugins-config-volume\") pod \"payload-processing-e2e-ait-09d8fb9b-7bc65f5f7f-hl4mj\" (UID: \"34f376f3-c14a-4736-9ce5-770d628e800e\") " pod="openshift-ingress/payload-processing-e2e-ait-09d8fb9b-7bc65f5f7f-hl4mj"
Oct 30 21:49:17.131354 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:17.131327    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34f376f3-c14a-4736-9ce5-770d628e800e-plugins-config-volume\") pod \"payload-processing-e2e-ait-09d8fb9b-7bc65f5f7f-hl4mj\" (UID: \"34f376f3-c14a-4736-9ce5-770d628e800e\") " pod="openshift-ingress/payload-processing-e2e-ait-09d8fb9b-7bc65f5f7f-hl4mj"
Oct 30 21:49:17.144017 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:17.143988    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-09d8fb9b-54956d9467-lncnc"
Oct 30 21:49:17.189775 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:17.189712    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t4h9\" (UniqueName: \"kubernetes.io/projected/34f376f3-c14a-4736-9ce5-770d628e800e-kube-api-access-5t4h9\") pod \"payload-processing-e2e-ait-09d8fb9b-7bc65f5f7f-hl4mj\" (UID: \"34f376f3-c14a-4736-9ce5-770d628e800e\") " pod="openshift-ingress/payload-processing-e2e-ait-09d8fb9b-7bc65f5f7f-hl4mj"
Oct 30 21:49:17.309531 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:17.309067    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-09d8fb9b-7bc65f5f7f-hl4mj"
Oct 30 21:49:17.527663 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:17.527633    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-09d8fb9b-54956d9467-lncnc"]
Oct 30 21:49:17.529304 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:49:17.529271    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57ee4e81_90ce_4670_8522_724918f6b037.slice/crio-30eb04e3d4c9abb65130702092f0e2ff9c0e31fc3b1803e8bf9f44100b7a8051 WatchSource:0}: Error finding container 30eb04e3d4c9abb65130702092f0e2ff9c0e31fc3b1803e8bf9f44100b7a8051: Status 404 returned error can't find the container with id 30eb04e3d4c9abb65130702092f0e2ff9c0e31fc3b1803e8bf9f44100b7a8051
Oct 30 21:49:17.701282 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:17.698663    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-09d8fb9b-7bc65f5f7f-hl4mj"]
Oct 30 21:49:18.467196 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:18.467163    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ebf644d4-6b5d7b4747-7jqqd" event={"ID":"452e61af-d766-4929-b607-a0ad2332b7cf","Type":"ContainerStarted","Data":"70411834ec3d339c845e14a5338c52f5db69334e8e445a87f2198ac768f05307"}
Oct 30 21:49:18.467581 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:18.467400    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-ebf644d4-6b5d7b4747-7jqqd"
Oct 30 21:49:18.468897 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:18.468869    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-09d8fb9b-7bc65f5f7f-hl4mj" event={"ID":"34f376f3-c14a-4736-9ce5-770d628e800e","Type":"ContainerStarted","Data":"8443e25344819cef90493fcb1f96904d670172501cb64d07907a3572ded12ee1"}
Oct 30 21:49:18.470197 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:18.470174    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-09d8fb9b-54956d9467-lncnc" event={"ID":"57ee4e81-90ce-4670-8522-724918f6b037","Type":"ContainerStarted","Data":"30eb04e3d4c9abb65130702092f0e2ff9c0e31fc3b1803e8bf9f44100b7a8051"}
Oct 30 21:49:18.581190 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:18.581140    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-ebf644d4-6b5d7b4747-7jqqd" podStartSLOduration=1.297550351 podStartE2EDuration="4.581122869s" totalImagesPullingTime="3.283572518s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:18.578620433 +0000 UTC m=+700.756109049" watchObservedRunningTime="2026-10-30 21:49:18.581122869 +0000 UTC m=+700.758611486"
Oct 30 21:49:19.023311 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:19.023271    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:19.201945 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:19.201900    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:19.422592 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:19.422548    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-16c49820-5654f658d7-v744c"]
Oct 30 21:49:19.426760 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:19.426733    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-16c49820-5654f658d7-v744c"
Oct 30 21:49:19.464262 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:19.464213    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5ce1b1a-89d4-4336-aee6-f71b2f8a0405-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-16c49820-5654f658d7-v744c\" (UID: \"b5ce1b1a-89d4-4336-aee6-f71b2f8a0405\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-16c49820-5654f658d7-v744c"
Oct 30 21:49:19.464441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:19.464327    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4fk7\" (UniqueName: \"kubernetes.io/projected/b5ce1b1a-89d4-4336-aee6-f71b2f8a0405-kube-api-access-j4fk7\") pod \"payload-pre-processing-e2e-migrate-16c49820-5654f658d7-v744c\" (UID: \"b5ce1b1a-89d4-4336-aee6-f71b2f8a0405\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-16c49820-5654f658d7-v744c"
Oct 30 21:49:19.477263 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:19.477183    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-16c49820\""
Oct 30 21:49:19.477764 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:19.477357    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-16c49820-dockercfg-gv48w\""
Oct 30 21:49:19.478863 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:19.478803    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-09d8fb9b-7bc65f5f7f-hl4mj" event={"ID":"34f376f3-c14a-4736-9ce5-770d628e800e","Type":"ContainerStarted","Data":"0a44aaf2e46030c003f7b2e56a0e981a370abd93066dab5b81a1d18463027b2c"}
Oct 30 21:49:19.479012 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:19.478932    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-09d8fb9b-7bc65f5f7f-hl4mj"
Oct 30 21:49:19.482021 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:19.481995    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-16c49820-5654f658d7-v744c"]
Oct 30 21:49:19.482419 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:19.482392    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-09d8fb9b-54956d9467-lncnc" event={"ID":"57ee4e81-90ce-4670-8522-724918f6b037","Type":"ContainerStarted","Data":"a56c467638f119f8e0d44a4ac780b2bf8945a60b97573aa97645c95b9b5cee85"}
Oct 30 21:49:19.482542 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:19.482480    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-09d8fb9b-54956d9467-lncnc"
Oct 30 21:49:19.565196 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:19.565124    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4fk7\" (UniqueName: \"kubernetes.io/projected/b5ce1b1a-89d4-4336-aee6-f71b2f8a0405-kube-api-access-j4fk7\") pod \"payload-pre-processing-e2e-migrate-16c49820-5654f658d7-v744c\" (UID: \"b5ce1b1a-89d4-4336-aee6-f71b2f8a0405\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-16c49820-5654f658d7-v744c"
Oct 30 21:49:19.565409 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:19.565323    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5ce1b1a-89d4-4336-aee6-f71b2f8a0405-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-16c49820-5654f658d7-v744c\" (UID: \"b5ce1b1a-89d4-4336-aee6-f71b2f8a0405\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-16c49820-5654f658d7-v744c"
Oct 30 21:49:19.566039 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:19.566018    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5ce1b1a-89d4-4336-aee6-f71b2f8a0405-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-16c49820-5654f658d7-v744c\" (UID: \"b5ce1b1a-89d4-4336-aee6-f71b2f8a0405\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-16c49820-5654f658d7-v744c"
Oct 30 21:49:19.582353 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:19.582314    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-16c49820-84664c54c-2592k"]
Oct 30 21:49:19.586269 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:19.586231    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-16c49820-84664c54c-2592k"
Oct 30 21:49:19.623773 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:19.623735    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4fk7\" (UniqueName: \"kubernetes.io/projected/b5ce1b1a-89d4-4336-aee6-f71b2f8a0405-kube-api-access-j4fk7\") pod \"payload-pre-processing-e2e-migrate-16c49820-5654f658d7-v744c\" (UID: \"b5ce1b1a-89d4-4336-aee6-f71b2f8a0405\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-16c49820-5654f658d7-v744c"
Oct 30 21:49:19.640925 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:19.640888    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-16c49820-84664c54c-2592k"]
Oct 30 21:49:19.651382 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:19.651329    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-09d8fb9b-54956d9467-lncnc" podStartSLOduration=2.73780599 podStartE2EDuration="3.651315294s" totalImagesPullingTime="913.509304ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:19.65002052 +0000 UTC m=+701.827509128" watchObservedRunningTime="2026-10-30 21:49:19.651315294 +0000 UTC m=+701.828803910"
Oct 30 21:49:19.666392 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:19.666355    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72192587-c584-4ed8-b949-a72df97551d9-plugins-config-volume\") pod \"payload-processing-e2e-migrate-16c49820-84664c54c-2592k\" (UID: \"72192587-c584-4ed8-b949-a72df97551d9\") " pod="openshift-ingress/payload-processing-e2e-migrate-16c49820-84664c54c-2592k"
Oct 30 21:49:19.666566 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:19.666486    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdl89\" (UniqueName: \"kubernetes.io/projected/72192587-c584-4ed8-b949-a72df97551d9-kube-api-access-gdl89\") pod \"payload-processing-e2e-migrate-16c49820-84664c54c-2592k\" (UID: \"72192587-c584-4ed8-b949-a72df97551d9\") " pod="openshift-ingress/payload-processing-e2e-migrate-16c49820-84664c54c-2592k"
Oct 30 21:49:19.740508 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:19.740416    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-16c49820-5654f658d7-v744c"
Oct 30 21:49:19.763153 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:19.763069    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-09d8fb9b-7bc65f5f7f-hl4mj" podStartSLOduration=2.959411702 podStartE2EDuration="3.763053666s" totalImagesPullingTime="803.641964ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:19.760471157 +0000 UTC m=+701.937959777" watchObservedRunningTime="2026-10-30 21:49:19.763053666 +0000 UTC m=+701.940542282"
Oct 30 21:49:19.767050 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:19.767018    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdl89\" (UniqueName: \"kubernetes.io/projected/72192587-c584-4ed8-b949-a72df97551d9-kube-api-access-gdl89\") pod \"payload-processing-e2e-migrate-16c49820-84664c54c-2592k\" (UID: \"72192587-c584-4ed8-b949-a72df97551d9\") " pod="openshift-ingress/payload-processing-e2e-migrate-16c49820-84664c54c-2592k"
Oct 30 21:49:19.767180 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:19.767146    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72192587-c584-4ed8-b949-a72df97551d9-plugins-config-volume\") pod \"payload-processing-e2e-migrate-16c49820-84664c54c-2592k\" (UID: \"72192587-c584-4ed8-b949-a72df97551d9\") " pod="openshift-ingress/payload-processing-e2e-migrate-16c49820-84664c54c-2592k"
Oct 30 21:49:19.767874 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:19.767847    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72192587-c584-4ed8-b949-a72df97551d9-plugins-config-volume\") pod \"payload-processing-e2e-migrate-16c49820-84664c54c-2592k\" (UID: \"72192587-c584-4ed8-b949-a72df97551d9\") " pod="openshift-ingress/payload-processing-e2e-migrate-16c49820-84664c54c-2592k"
Oct 30 21:49:19.826003 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:19.825968    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdl89\" (UniqueName: \"kubernetes.io/projected/72192587-c584-4ed8-b949-a72df97551d9-kube-api-access-gdl89\") pod \"payload-processing-e2e-migrate-16c49820-84664c54c-2592k\" (UID: \"72192587-c584-4ed8-b949-a72df97551d9\") " pod="openshift-ingress/payload-processing-e2e-migrate-16c49820-84664c54c-2592k"
Oct 30 21:49:19.901264 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:19.900122    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-16c49820-84664c54c-2592k"
Oct 30 21:49:20.114925 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:20.114889    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-16c49820-5654f658d7-v744c"]
Oct 30 21:49:20.116740 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:49:20.116712    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5ce1b1a_89d4_4336_aee6_f71b2f8a0405.slice/crio-e1ed970c8573887c8d8f6f43d659fb99c3b366fb0c1f9f42b2846aa38bf083d6 WatchSource:0}: Error finding container e1ed970c8573887c8d8f6f43d659fb99c3b366fb0c1f9f42b2846aa38bf083d6: Status 404 returned error can't find the container with id e1ed970c8573887c8d8f6f43d659fb99c3b366fb0c1f9f42b2846aa38bf083d6
Oct 30 21:49:20.280193 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:20.280161    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-16c49820-84664c54c-2592k"]
Oct 30 21:49:20.281713 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:49:20.281683    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72192587_c584_4ed8_b949_a72df97551d9.slice/crio-3da9ee38dcd0354b5fc6b6d489691eac6fed8b1859e2093d0f33ceac8325f869 WatchSource:0}: Error finding container 3da9ee38dcd0354b5fc6b6d489691eac6fed8b1859e2093d0f33ceac8325f869: Status 404 returned error can't find the container with id 3da9ee38dcd0354b5fc6b6d489691eac6fed8b1859e2093d0f33ceac8325f869
Oct 30 21:49:20.492172 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:20.492073    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-16c49820-84664c54c-2592k" event={"ID":"72192587-c584-4ed8-b949-a72df97551d9","Type":"ContainerStarted","Data":"2717cc0c635b383dcd947d57ba11a246f5b39bf9afc51a5162e44318aae1090d"}
Oct 30 21:49:20.492172 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:20.492171    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-16c49820-84664c54c-2592k"
Oct 30 21:49:20.492742 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:20.492190    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-16c49820-84664c54c-2592k" event={"ID":"72192587-c584-4ed8-b949-a72df97551d9","Type":"ContainerStarted","Data":"3da9ee38dcd0354b5fc6b6d489691eac6fed8b1859e2093d0f33ceac8325f869"}
Oct 30 21:49:20.493991 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:20.493953    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-16c49820-5654f658d7-v744c" event={"ID":"b5ce1b1a-89d4-4336-aee6-f71b2f8a0405","Type":"ContainerStarted","Data":"bf16263f0cd06627ce28336b8c6c40cee7594da2b926be22b2964da22d88f8da"}
Oct 30 21:49:20.494132 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:20.494002    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-16c49820-5654f658d7-v744c" event={"ID":"b5ce1b1a-89d4-4336-aee6-f71b2f8a0405","Type":"ContainerStarted","Data":"e1ed970c8573887c8d8f6f43d659fb99c3b366fb0c1f9f42b2846aa38bf083d6"}
Oct 30 21:49:20.494132 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:20.494117    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-16c49820-5654f658d7-v744c"
Oct 30 21:49:20.603323 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:20.603269    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-16c49820-84664c54c-2592k" podStartSLOduration=1.6032567659999999 podStartE2EDuration="1.603256766s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:20.601205242 +0000 UTC m=+702.778693859" watchObservedRunningTime="2026-10-30 21:49:20.603256766 +0000 UTC m=+702.780745380"
Oct 30 21:49:20.713763 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:20.713705    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-16c49820-5654f658d7-v744c" podStartSLOduration=1.713689936 podStartE2EDuration="1.713689936s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:20.710469824 +0000 UTC m=+702.887958442" watchObservedRunningTime="2026-10-30 21:49:20.713689936 +0000 UTC m=+702.891178590"
Oct 30 21:49:21.954600 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:21.954558    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:22.112687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:22.112640    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:22.262328 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:22.262223    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:26.907352 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:26.907312    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:28.815979 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:28.815941    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:29.035536 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:29.035499    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:29.377629 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:29.377586    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-ffe7a1cf-6976c8c597-t5ncj"]
Oct 30 21:49:29.382065 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:29.382034    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-ffe7a1cf-6976c8c597-t5ncj"
Oct 30 21:49:29.432796 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:29.432760    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-ffe7a1cf-6976c8c597-t5ncj"]
Oct 30 21:49:29.457258 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:29.457200    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4jnt\" (UniqueName: \"kubernetes.io/projected/a11db216-47ce-442a-adb9-ca7e682eb87e-kube-api-access-f4jnt\") pod \"payload-processing-e2e-derive-ffe7a1cf-6976c8c597-t5ncj\" (UID: \"a11db216-47ce-442a-adb9-ca7e682eb87e\") " pod="openshift-ingress/payload-processing-e2e-derive-ffe7a1cf-6976c8c597-t5ncj"
Oct 30 21:49:29.457493 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:29.457460    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a11db216-47ce-442a-adb9-ca7e682eb87e-plugins-config-volume\") pod \"payload-processing-e2e-derive-ffe7a1cf-6976c8c597-t5ncj\" (UID: \"a11db216-47ce-442a-adb9-ca7e682eb87e\") " pod="openshift-ingress/payload-processing-e2e-derive-ffe7a1cf-6976c8c597-t5ncj"
Oct 30 21:49:29.476126 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:29.476095    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-ffe7a1cf-dockercfg-22jxt\""
Oct 30 21:49:29.485994 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:29.485953    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-ebf644d4-6b5d7b4747-7jqqd"
Oct 30 21:49:29.495624 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:29.495600    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-ffe7a1cf\""
Oct 30 21:49:29.558661 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:29.558631    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a11db216-47ce-442a-adb9-ca7e682eb87e-plugins-config-volume\") pod \"payload-processing-e2e-derive-ffe7a1cf-6976c8c597-t5ncj\" (UID: \"a11db216-47ce-442a-adb9-ca7e682eb87e\") " pod="openshift-ingress/payload-processing-e2e-derive-ffe7a1cf-6976c8c597-t5ncj"
Oct 30 21:49:29.558844 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:29.558691    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4jnt\" (UniqueName: \"kubernetes.io/projected/a11db216-47ce-442a-adb9-ca7e682eb87e-kube-api-access-f4jnt\") pod \"payload-processing-e2e-derive-ffe7a1cf-6976c8c597-t5ncj\" (UID: \"a11db216-47ce-442a-adb9-ca7e682eb87e\") " pod="openshift-ingress/payload-processing-e2e-derive-ffe7a1cf-6976c8c597-t5ncj"
Oct 30 21:49:29.559726 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:29.559701    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a11db216-47ce-442a-adb9-ca7e682eb87e-plugins-config-volume\") pod \"payload-processing-e2e-derive-ffe7a1cf-6976c8c597-t5ncj\" (UID: \"a11db216-47ce-442a-adb9-ca7e682eb87e\") " pod="openshift-ingress/payload-processing-e2e-derive-ffe7a1cf-6976c8c597-t5ncj"
Oct 30 21:49:29.619215 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:29.619180    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4jnt\" (UniqueName: \"kubernetes.io/projected/a11db216-47ce-442a-adb9-ca7e682eb87e-kube-api-access-f4jnt\") pod \"payload-processing-e2e-derive-ffe7a1cf-6976c8c597-t5ncj\" (UID: \"a11db216-47ce-442a-adb9-ca7e682eb87e\") " pod="openshift-ingress/payload-processing-e2e-derive-ffe7a1cf-6976c8c597-t5ncj"
Oct 30 21:49:29.696276 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:29.696179    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-ffe7a1cf-6976c8c597-t5ncj"
Oct 30 21:49:30.079174 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:30.079146    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-ffe7a1cf-6976c8c597-t5ncj"]
Oct 30 21:49:30.081456 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:49:30.081425    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda11db216_47ce_442a_adb9_ca7e682eb87e.slice/crio-909b029ab4106eecb80534d50bdc9c58d42b789fec47f312b21135ef410cb28c WatchSource:0}: Error finding container 909b029ab4106eecb80534d50bdc9c58d42b789fec47f312b21135ef410cb28c: Status 404 returned error can't find the container with id 909b029ab4106eecb80534d50bdc9c58d42b789fec47f312b21135ef410cb28c
Oct 30 21:49:30.497641 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:30.497614    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-09d8fb9b-54956d9467-lncnc"
Oct 30 21:49:30.497814 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:30.497678    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-09d8fb9b-7bc65f5f7f-hl4mj"
Oct 30 21:49:30.562055 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:30.562019    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-ffe7a1cf-6976c8c597-t5ncj" event={"ID":"a11db216-47ce-442a-adb9-ca7e682eb87e","Type":"ContainerStarted","Data":"113a90e81e3ab67714ef34fc605d174c66190378e3bc931c99ef45f7b66f09eb"}
Oct 30 21:49:30.562223 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:30.562075    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-ffe7a1cf-6976c8c597-t5ncj" event={"ID":"a11db216-47ce-442a-adb9-ca7e682eb87e","Type":"ContainerStarted","Data":"909b029ab4106eecb80534d50bdc9c58d42b789fec47f312b21135ef410cb28c"}
Oct 30 21:49:30.562223 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:30.562099    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-ffe7a1cf-6976c8c597-t5ncj"
Oct 30 21:49:30.830294 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:30.830161    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-ffe7a1cf-6976c8c597-t5ncj" podStartSLOduration=1.830134345 podStartE2EDuration="1.830134345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:30.82728967 +0000 UTC m=+713.004778287" watchObservedRunningTime="2026-10-30 21:49:30.830134345 +0000 UTC m=+713.007622959"
Oct 30 21:49:31.449817 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:31.449784    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:31.505366 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:31.505334    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-16c49820-84664c54c-2592k"
Oct 30 21:49:31.505609 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:31.505423    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-16c49820-5654f658d7-v744c"
Oct 30 21:49:31.607042 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:31.607003    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:31.760675 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:31.760379    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:35.568560 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:49:35.568516    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:49:35.889459 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:35.889424    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:36.043614 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:36.043576    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:36.188307 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:36.188158    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:40.794872 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:40.792494    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:41.571770 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:41.571737    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-ffe7a1cf-6976c8c597-t5ncj"
Oct 30 21:49:43.679564 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:43.679527    4871 generic.go:356] "Generic (PLEG): container finished" podID="d11e9475-1d27-44b3-b433-13a5cc49a13a" containerID="948826c218be85302c71f09ea7e45d8636ac493c23c162e3f1d7c4a1cfcf6c75" exitCode=0
Oct 30 21:49:43.680068 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:43.679601    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw" event={"ID":"d11e9475-1d27-44b3-b433-13a5cc49a13a","Type":"ContainerDied","Data":"948826c218be85302c71f09ea7e45d8636ac493c23c162e3f1d7c4a1cfcf6c75"}
Oct 30 21:49:43.680068 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:43.679989    4871 scope.go:122] "RemoveContainer" containerID="948826c218be85302c71f09ea7e45d8636ac493c23c162e3f1d7c4a1cfcf6c75"
Oct 30 21:49:43.681250 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:43.681211    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:49:43.681320 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:43.681307    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:49:43.681364 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:43.681353    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:49:44.688781 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:44.688743    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw" event={"ID":"d11e9475-1d27-44b3-b433-13a5cc49a13a","Type":"ContainerStarted","Data":"bee8fa33b605bcadb1c43011eb60ffe6a41f4c9d75be42d381c5721e9f02c800"}
Oct 30 21:49:45.329014 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:45.328979    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:45.494140 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:45.494106    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"
Oct 30 21:49:45.495181 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:45.495155    4871 patch_prober.go:29] interesting pod/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 30 21:49:45.495316 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:45.495209    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw" podUID="d11e9475-1d27-44b3-b433-13a5cc49a13a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 30 21:49:46.493947 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:46.493902    4871 patch_prober.go:29] interesting pod/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 30 21:49:46.494511 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:46.493967    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw" podUID="d11e9475-1d27-44b3-b433-13a5cc49a13a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 30 21:49:47.494147 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:47.494099    4871 patch_prober.go:29] interesting pod/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 30 21:49:47.494589 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:47.494177    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw" podUID="d11e9475-1d27-44b3-b433-13a5cc49a13a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 30 21:49:48.033144 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:48.033104    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ebf644d4-rppbl"]
Oct 30 21:49:48.041392 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:48.041361    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ebf644d4-rppbl"
Oct 30 21:49:48.084640 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:48.084599    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ebf644d4-rppbl"]
Oct 30 21:49:48.087062 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:48.087029    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/53f636b4-eeb0-4a65-a391-965a83079670-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-ebf644d4-rppbl\" (UID: \"53f636b4-eeb0-4a65-a391-965a83079670\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ebf644d4-rppbl"
Oct 30 21:49:48.087226 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:48.087110    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f574j\" (UniqueName: \"kubernetes.io/projected/53f636b4-eeb0-4a65-a391-965a83079670-kube-api-access-f574j\") pod \"maas-api-revoke-keys-e2e-ait-ebf644d4-rppbl\" (UID: \"53f636b4-eeb0-4a65-a391-965a83079670\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ebf644d4-rppbl"
Oct 30 21:49:48.087226 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:48.087158    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/53f636b4-eeb0-4a65-a391-965a83079670-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-ebf644d4-rppbl\" (UID: \"53f636b4-eeb0-4a65-a391-965a83079670\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ebf644d4-rppbl"
Oct 30 21:49:48.133823 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:48.133674    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:49:48.133823 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:48.133713    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 21:49:48.137291 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:48.137201    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mlvtf\""
Oct 30 21:49:48.137483 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:48.137325    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 21:49:48.188387 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:48.188339    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/53f636b4-eeb0-4a65-a391-965a83079670-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-ebf644d4-rppbl\" (UID: \"53f636b4-eeb0-4a65-a391-965a83079670\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ebf644d4-rppbl"
Oct 30 21:49:48.188593 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:48.188435    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f574j\" (UniqueName: \"kubernetes.io/projected/53f636b4-eeb0-4a65-a391-965a83079670-kube-api-access-f574j\") pod \"maas-api-revoke-keys-e2e-ait-ebf644d4-rppbl\" (UID: \"53f636b4-eeb0-4a65-a391-965a83079670\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ebf644d4-rppbl"
Oct 30 21:49:48.188593 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:48.188455    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/53f636b4-eeb0-4a65-a391-965a83079670-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-ebf644d4-rppbl\" (UID: \"53f636b4-eeb0-4a65-a391-965a83079670\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ebf644d4-rppbl"
Oct 30 21:49:48.189315 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:48.189271    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/53f636b4-eeb0-4a65-a391-965a83079670-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-ebf644d4-rppbl\" (UID: \"53f636b4-eeb0-4a65-a391-965a83079670\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ebf644d4-rppbl"
Oct 30 21:49:48.190849 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:48.190825    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/53f636b4-eeb0-4a65-a391-965a83079670-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-ebf644d4-rppbl\" (UID: \"53f636b4-eeb0-4a65-a391-965a83079670\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ebf644d4-rppbl"
Oct 30 21:49:48.246828 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:48.246791    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f574j\" (UniqueName: \"kubernetes.io/projected/53f636b4-eeb0-4a65-a391-965a83079670-kube-api-access-f574j\") pod \"maas-api-revoke-keys-e2e-ait-ebf644d4-rppbl\" (UID: \"53f636b4-eeb0-4a65-a391-965a83079670\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ebf644d4-rppbl"
Oct 30 21:49:48.357785 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:48.357701    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ebf644d4-rppbl"
Oct 30 21:49:48.494417 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:48.494383    4871 patch_prober.go:29] interesting pod/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 30 21:49:48.494811 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:48.494442    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw" podUID="d11e9475-1d27-44b3-b433-13a5cc49a13a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 30 21:49:48.760330 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:48.760281    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ebf644d4-rppbl"]
Oct 30 21:49:48.762192 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:49:48.762155    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53f636b4_eeb0_4a65_a391_965a83079670.slice/crio-db37a906d3bb801670ee7f89c279c739d3fcfb74c9cbfa07a0b4cc6588af6e40 WatchSource:0}: Error finding container db37a906d3bb801670ee7f89c279c739d3fcfb74c9cbfa07a0b4cc6588af6e40: Status 404 returned error can't find the container with id db37a906d3bb801670ee7f89c279c739d3fcfb74c9cbfa07a0b4cc6588af6e40
Oct 30 21:49:49.348432 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:49.348393    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"
Oct 30 21:49:49.496655 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:49.496617    4871 patch_prober.go:29] interesting pod/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 30 21:49:49.497212 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:49.497179    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw" podUID="d11e9475-1d27-44b3-b433-13a5cc49a13a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 30 21:49:49.755391 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:49.755344    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ebf644d4-rppbl" event={"ID":"53f636b4-eeb0-4a65-a391-965a83079670","Type":"ContainerStarted","Data":"db37a906d3bb801670ee7f89c279c739d3fcfb74c9cbfa07a0b4cc6588af6e40"}
Oct 30 21:49:50.498294 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:50.498265    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"
Oct 30 21:49:50.499252 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:50.499218    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"
Oct 30 21:49:50.538639 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:50.538597    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:53.786061 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:53.786008    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ebf644d4-rppbl" event={"ID":"53f636b4-eeb0-4a65-a391-965a83079670","Type":"ContainerStarted","Data":"9b412d2dcd5731955776dade6ab0a638a312bc13eddb208b78c9e183c38d6af1"}
Oct 30 21:49:53.895013 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:53.894944    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ebf644d4-rppbl" podStartSLOduration=2.606323738 podStartE2EDuration="6.894928325s" totalImagesPullingTime="4.288604587s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:53.894605195 +0000 UTC m=+736.072093815" watchObservedRunningTime="2026-10-30 21:49:53.894928325 +0000 UTC m=+736.072416942"
Oct 30 21:49:54.796166 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:54.796121    4871 generic.go:356] "Generic (PLEG): container finished" podID="53f636b4-eeb0-4a65-a391-965a83079670" containerID="9b412d2dcd5731955776dade6ab0a638a312bc13eddb208b78c9e183c38d6af1" exitCode=0
Oct 30 21:49:54.796695 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:54.796206    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ebf644d4-rppbl" event={"ID":"53f636b4-eeb0-4a65-a391-965a83079670","Type":"ContainerDied","Data":"9b412d2dcd5731955776dade6ab0a638a312bc13eddb208b78c9e183c38d6af1"}
Oct 30 21:49:55.022480 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:55.022435    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-09d8fb9b-thlrl"]
Oct 30 21:49:55.037025 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:55.036990    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-09d8fb9b-thlrl"
Oct 30 21:49:55.077157 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:55.077070    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-09d8fb9b-thlrl"]
Oct 30 21:49:55.192219 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:55.192182    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a9dfd1e8-cfd1-414a-92d5-1241a669d360-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-09d8fb9b-thlrl\" (UID: \"a9dfd1e8-cfd1-414a-92d5-1241a669d360\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-09d8fb9b-thlrl"
Oct 30 21:49:55.192420 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:55.192322    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a9dfd1e8-cfd1-414a-92d5-1241a669d360-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-09d8fb9b-thlrl\" (UID: \"a9dfd1e8-cfd1-414a-92d5-1241a669d360\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-09d8fb9b-thlrl"
Oct 30 21:49:55.192420 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:55.192402    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5gzl\" (UniqueName: \"kubernetes.io/projected/a9dfd1e8-cfd1-414a-92d5-1241a669d360-kube-api-access-r5gzl\") pod \"maas-api-revoke-keys-e2e-ait-09d8fb9b-thlrl\" (UID: \"a9dfd1e8-cfd1-414a-92d5-1241a669d360\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-09d8fb9b-thlrl"
Oct 30 21:49:55.293452 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:55.293413    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a9dfd1e8-cfd1-414a-92d5-1241a669d360-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-09d8fb9b-thlrl\" (UID: \"a9dfd1e8-cfd1-414a-92d5-1241a669d360\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-09d8fb9b-thlrl"
Oct 30 21:49:55.293673 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:55.293557    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5gzl\" (UniqueName: \"kubernetes.io/projected/a9dfd1e8-cfd1-414a-92d5-1241a669d360-kube-api-access-r5gzl\") pod \"maas-api-revoke-keys-e2e-ait-09d8fb9b-thlrl\" (UID: \"a9dfd1e8-cfd1-414a-92d5-1241a669d360\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-09d8fb9b-thlrl"
Oct 30 21:49:55.293673 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:55.293630    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a9dfd1e8-cfd1-414a-92d5-1241a669d360-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-09d8fb9b-thlrl\" (UID: \"a9dfd1e8-cfd1-414a-92d5-1241a669d360\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-09d8fb9b-thlrl"
Oct 30 21:49:55.294301 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:55.294278    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a9dfd1e8-cfd1-414a-92d5-1241a669d360-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-09d8fb9b-thlrl\" (UID: \"a9dfd1e8-cfd1-414a-92d5-1241a669d360\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-09d8fb9b-thlrl"
Oct 30 21:49:55.295924 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:55.295901    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a9dfd1e8-cfd1-414a-92d5-1241a669d360-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-09d8fb9b-thlrl\" (UID: \"a9dfd1e8-cfd1-414a-92d5-1241a669d360\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-09d8fb9b-thlrl"
Oct 30 21:49:55.351979 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:55.351894    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5gzl\" (UniqueName: \"kubernetes.io/projected/a9dfd1e8-cfd1-414a-92d5-1241a669d360-kube-api-access-r5gzl\") pod \"maas-api-revoke-keys-e2e-ait-09d8fb9b-thlrl\" (UID: \"a9dfd1e8-cfd1-414a-92d5-1241a669d360\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-09d8fb9b-thlrl"
Oct 30 21:49:55.564341 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:55.564288    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:55.656469 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:55.656434    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-09d8fb9b-thlrl"
Oct 30 21:49:56.014503 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:56.014462    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ebf644d4-rppbl"
Oct 30 21:49:56.057120 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:56.057089    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-09d8fb9b-thlrl"]
Oct 30 21:49:56.057845 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:49:56.057822    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9dfd1e8_cfd1_414a_92d5_1241a669d360.slice/crio-7b524235d0557124dab53597bba8d78dcc360ae4cc8ab0c770789e6303492ebf WatchSource:0}: Error finding container 7b524235d0557124dab53597bba8d78dcc360ae4cc8ab0c770789e6303492ebf: Status 404 returned error can't find the container with id 7b524235d0557124dab53597bba8d78dcc360ae4cc8ab0c770789e6303492ebf
Oct 30 21:49:56.122882 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:56.121542    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/53f636b4-eeb0-4a65-a391-965a83079670-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/53f636b4-eeb0-4a65-a391-965a83079670-maas-api-xks-ca\") pod \"53f636b4-eeb0-4a65-a391-965a83079670\" (UID: \"53f636b4-eeb0-4a65-a391-965a83079670\") "
Oct 30 21:49:56.122882 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:56.121721    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/53f636b4-eeb0-4a65-a391-965a83079670-kube-api-access-f574j\" (UniqueName: \"kubernetes.io/projected/53f636b4-eeb0-4a65-a391-965a83079670-kube-api-access-f574j\") pod \"53f636b4-eeb0-4a65-a391-965a83079670\" (UID: \"53f636b4-eeb0-4a65-a391-965a83079670\") "
Oct 30 21:49:56.122882 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:56.121760    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/53f636b4-eeb0-4a65-a391-965a83079670-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/53f636b4-eeb0-4a65-a391-965a83079670-maas-api-openshift-ca\") pod \"53f636b4-eeb0-4a65-a391-965a83079670\" (UID: \"53f636b4-eeb0-4a65-a391-965a83079670\") "
Oct 30 21:49:56.124109 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:56.123839    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53f636b4-eeb0-4a65-a391-965a83079670-maas-api-openshift-ca" pod "53f636b4-eeb0-4a65-a391-965a83079670" (UID: "53f636b4-eeb0-4a65-a391-965a83079670"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:56.129715 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:56.129658    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53f636b4-eeb0-4a65-a391-965a83079670-maas-api-xks-ca" pod "53f636b4-eeb0-4a65-a391-965a83079670" (UID: "53f636b4-eeb0-4a65-a391-965a83079670"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:49:56.131878 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:56.131799    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53f636b4-eeb0-4a65-a391-965a83079670-kube-api-access-f574j" pod "53f636b4-eeb0-4a65-a391-965a83079670" (UID: "53f636b4-eeb0-4a65-a391-965a83079670"). InnerVolumeSpecName "kube-api-access-f574j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:56.222866 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:56.222837    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f574j\" (UniqueName: \"kubernetes.io/projected/53f636b4-eeb0-4a65-a391-965a83079670-kube-api-access-f574j\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:56.223050 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:56.223038    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/53f636b4-eeb0-4a65-a391-965a83079670-maas-api-openshift-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:56.223116 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:56.223108    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/53f636b4-eeb0-4a65-a391-965a83079670-maas-api-xks-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:56.811874 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:56.811837    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-09d8fb9b-thlrl" event={"ID":"a9dfd1e8-cfd1-414a-92d5-1241a669d360","Type":"ContainerStarted","Data":"a0fb0f1bd6a507efbd5b58abec574a38ffad9ce1aaa01a9acb8ae7ea75284031"}
Oct 30 21:49:56.812060 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:56.811901    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-09d8fb9b-thlrl" event={"ID":"a9dfd1e8-cfd1-414a-92d5-1241a669d360","Type":"ContainerStarted","Data":"7b524235d0557124dab53597bba8d78dcc360ae4cc8ab0c770789e6303492ebf"}
Oct 30 21:49:56.813317 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:56.813295    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ebf644d4-rppbl"
Oct 30 21:49:56.813427 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:56.813337    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ebf644d4-rppbl" event={"ID":"53f636b4-eeb0-4a65-a391-965a83079670","Type":"ContainerDied","Data":"db37a906d3bb801670ee7f89c279c739d3fcfb74c9cbfa07a0b4cc6588af6e40"}
Oct 30 21:49:56.813427 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:56.813382    4871 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="db37a906d3bb801670ee7f89c279c739d3fcfb74c9cbfa07a0b4cc6588af6e40"
Oct 30 21:49:56.921541 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:56.921477    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-09d8fb9b-thlrl" podStartSLOduration=2.921464029 podStartE2EDuration="2.921464029s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:56.920729262 +0000 UTC m=+739.098217879" watchObservedRunningTime="2026-10-30 21:49:56.921464029 +0000 UTC m=+739.098952646"
Oct 30 21:49:57.821880 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:57.821846    4871 generic.go:356] "Generic (PLEG): container finished" podID="a9dfd1e8-cfd1-414a-92d5-1241a669d360" containerID="a0fb0f1bd6a507efbd5b58abec574a38ffad9ce1aaa01a9acb8ae7ea75284031" exitCode=0
Oct 30 21:49:57.822408 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:57.821933    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-09d8fb9b-thlrl" event={"ID":"a9dfd1e8-cfd1-414a-92d5-1241a669d360","Type":"ContainerDied","Data":"a0fb0f1bd6a507efbd5b58abec574a38ffad9ce1aaa01a9acb8ae7ea75284031"}
Oct 30 21:49:58.465438 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:58.465392    4871 patch_prober.go:29] interesting pod/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:49:58.465638 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:58.465459    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg" podUID="03d16436-58d3-495e-84d4-38b6d3b55a8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:49:58.499205 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:58.499168    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ebf644d4-6b5d7b4747-7jqqd"]
Oct 30 21:49:58.783897 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:58.783806    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:58.975127 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:58.975025    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:59.052988 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:59.052910    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-09d8fb9b-thlrl"
Oct 30 21:49:59.171230 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:59.171192    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9dfd1e8-cfd1-414a-92d5-1241a669d360-kube-api-access-r5gzl\" (UniqueName: \"kubernetes.io/projected/a9dfd1e8-cfd1-414a-92d5-1241a669d360-kube-api-access-r5gzl\") pod \"a9dfd1e8-cfd1-414a-92d5-1241a669d360\" (UID: \"a9dfd1e8-cfd1-414a-92d5-1241a669d360\") "
Oct 30 21:49:59.171545 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:59.171518    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a9dfd1e8-cfd1-414a-92d5-1241a669d360-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a9dfd1e8-cfd1-414a-92d5-1241a669d360-maas-api-xks-ca\") pod \"a9dfd1e8-cfd1-414a-92d5-1241a669d360\" (UID: \"a9dfd1e8-cfd1-414a-92d5-1241a669d360\") "
Oct 30 21:49:59.171799 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:59.171777    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a9dfd1e8-cfd1-414a-92d5-1241a669d360-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a9dfd1e8-cfd1-414a-92d5-1241a669d360-maas-api-openshift-ca\") pod \"a9dfd1e8-cfd1-414a-92d5-1241a669d360\" (UID: \"a9dfd1e8-cfd1-414a-92d5-1241a669d360\") "
Oct 30 21:49:59.172298 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:59.172268    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9dfd1e8-cfd1-414a-92d5-1241a669d360-maas-api-openshift-ca" pod "a9dfd1e8-cfd1-414a-92d5-1241a669d360" (UID: "a9dfd1e8-cfd1-414a-92d5-1241a669d360"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:59.175629 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:59.175601    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9dfd1e8-cfd1-414a-92d5-1241a669d360-maas-api-xks-ca" pod "a9dfd1e8-cfd1-414a-92d5-1241a669d360" (UID: "a9dfd1e8-cfd1-414a-92d5-1241a669d360"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:49:59.175817 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:59.175793    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9dfd1e8-cfd1-414a-92d5-1241a669d360-kube-api-access-r5gzl" pod "a9dfd1e8-cfd1-414a-92d5-1241a669d360" (UID: "a9dfd1e8-cfd1-414a-92d5-1241a669d360"). InnerVolumeSpecName "kube-api-access-r5gzl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:59.273407 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:59.273365    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a9dfd1e8-cfd1-414a-92d5-1241a669d360-maas-api-openshift-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:59.273407 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:59.273410    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5gzl\" (UniqueName: \"kubernetes.io/projected/a9dfd1e8-cfd1-414a-92d5-1241a669d360-kube-api-access-r5gzl\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:59.273629 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:59.273428    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a9dfd1e8-cfd1-414a-92d5-1241a669d360-maas-api-xks-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:59.896223 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:59.896185    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-09d8fb9b-thlrl" event={"ID":"a9dfd1e8-cfd1-414a-92d5-1241a669d360","Type":"ContainerDied","Data":"7b524235d0557124dab53597bba8d78dcc360ae4cc8ab0c770789e6303492ebf"}
Oct 30 21:49:59.896509 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:59.896281    4871 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7b524235d0557124dab53597bba8d78dcc360ae4cc8ab0c770789e6303492ebf"
Oct 30 21:49:59.896509 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:59.896359    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-09d8fb9b-thlrl"
Oct 30 21:49:59.899996 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:59.899962    4871 generic.go:356] "Generic (PLEG): container finished" podID="03d16436-58d3-495e-84d4-38b6d3b55a8b" containerID="ded087dbe4633a08e6dbfe18cf8f74717787fd47a7044fda7a3cb81004168ee5" exitCode=0
Oct 30 21:49:59.900167 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:59.900108    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg" event={"ID":"03d16436-58d3-495e-84d4-38b6d3b55a8b","Type":"ContainerDied","Data":"ded087dbe4633a08e6dbfe18cf8f74717787fd47a7044fda7a3cb81004168ee5"}
Oct 30 21:49:59.901185 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:59.900539    4871 scope.go:122] "RemoveContainer" containerID="ded087dbe4633a08e6dbfe18cf8f74717787fd47a7044fda7a3cb81004168ee5"
Oct 30 21:49:59.902585 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:59.902534    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:49:59.902665 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:59.902629    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:49:59.902723 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:49:59.902704    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:50:00.445500 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:00.445457    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:00.932016 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:00.931980    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg" event={"ID":"03d16436-58d3-495e-84d4-38b6d3b55a8b","Type":"ContainerStarted","Data":"6cf1c83712e25baed3812e82ef12b839c26496846ddb9ae779273fdd266a9f30"}
Oct 30 21:50:01.871660 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:01.871621    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"
Oct 30 21:50:01.872093 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:01.871863    4871 patch_prober.go:29] interesting pod/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Oct 30 21:50:01.872093 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:01.871943    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg" podUID="03d16436-58d3-495e-84d4-38b6d3b55a8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Oct 30 21:50:02.433996 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:02.433955    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1d107b0-5f8m5"]
Oct 30 21:50:02.434474 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:02.434458    4871 cpu_manager.go:414] "containerMap: removing container" podUID="a9dfd1e8-cfd1-414a-92d5-1241a669d360" containerName="revoke-keys"
Oct 30 21:50:02.434528 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:02.434475    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9dfd1e8-cfd1-414a-92d5-1241a669d360" containerName="revoke-keys"
Oct 30 21:50:02.434528 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:02.434491    4871 cpu_manager.go:414] "containerMap: removing container" podUID="53f636b4-eeb0-4a65-a391-965a83079670" containerName="revoke-keys"
Oct 30 21:50:02.434528 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:02.434496    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="53f636b4-eeb0-4a65-a391-965a83079670" containerName="revoke-keys"
Oct 30 21:50:02.434620 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:02.434597    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="a9dfd1e8-cfd1-414a-92d5-1241a669d360" containerName="revoke-keys"
Oct 30 21:50:02.434620 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:02.434608    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="53f636b4-eeb0-4a65-a391-965a83079670" containerName="revoke-keys"
Oct 30 21:50:02.437169 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:02.437149    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1d107b0-5f8m5"
Oct 30 21:50:02.439918 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:02.439892    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a752c8be-55df-4838-b95e-3a643323142b-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-a1d107b0-5f8m5\" (UID: \"a752c8be-55df-4838-b95e-3a643323142b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1d107b0-5f8m5"
Oct 30 21:50:02.440005 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:02.439942    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a752c8be-55df-4838-b95e-3a643323142b-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-a1d107b0-5f8m5\" (UID: \"a752c8be-55df-4838-b95e-3a643323142b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1d107b0-5f8m5"
Oct 30 21:50:02.440051 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:02.440012    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8ncc\" (UniqueName: \"kubernetes.io/projected/a752c8be-55df-4838-b95e-3a643323142b-kube-api-access-w8ncc\") pod \"maas-api-revoke-keys-e2e-ait-a1d107b0-5f8m5\" (UID: \"a752c8be-55df-4838-b95e-3a643323142b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1d107b0-5f8m5"
Oct 30 21:50:02.487011 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:02.486978    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 21:50:02.487207 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:02.487114    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 21:50:02.533653 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:02.533611    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:50:02.533866 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:02.533812    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mlvtf\""
Oct 30 21:50:02.537169 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:02.537140    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1d107b0-5f8m5"]
Oct 30 21:50:02.540574 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:02.540555    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a752c8be-55df-4838-b95e-3a643323142b-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-a1d107b0-5f8m5\" (UID: \"a752c8be-55df-4838-b95e-3a643323142b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1d107b0-5f8m5"
Oct 30 21:50:02.540654 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:02.540592    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8ncc\" (UniqueName: \"kubernetes.io/projected/a752c8be-55df-4838-b95e-3a643323142b-kube-api-access-w8ncc\") pod \"maas-api-revoke-keys-e2e-ait-a1d107b0-5f8m5\" (UID: \"a752c8be-55df-4838-b95e-3a643323142b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1d107b0-5f8m5"
Oct 30 21:50:02.540692 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:02.540661    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a752c8be-55df-4838-b95e-3a643323142b-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-a1d107b0-5f8m5\" (UID: \"a752c8be-55df-4838-b95e-3a643323142b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1d107b0-5f8m5"
Oct 30 21:50:02.541395 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:02.541373    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a752c8be-55df-4838-b95e-3a643323142b-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-a1d107b0-5f8m5\" (UID: \"a752c8be-55df-4838-b95e-3a643323142b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1d107b0-5f8m5"
Oct 30 21:50:02.542918 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:02.542900    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a752c8be-55df-4838-b95e-3a643323142b-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-a1d107b0-5f8m5\" (UID: \"a752c8be-55df-4838-b95e-3a643323142b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1d107b0-5f8m5"
Oct 30 21:50:02.628065 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:02.628020    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8ncc\" (UniqueName: \"kubernetes.io/projected/a752c8be-55df-4838-b95e-3a643323142b-kube-api-access-w8ncc\") pod \"maas-api-revoke-keys-e2e-ait-a1d107b0-5f8m5\" (UID: \"a752c8be-55df-4838-b95e-3a643323142b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1d107b0-5f8m5"
Oct 30 21:50:02.750969 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:02.750875    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1d107b0-5f8m5"
Oct 30 21:50:02.872247 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:02.872197    4871 patch_prober.go:29] interesting pod/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Oct 30 21:50:02.872617 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:02.872263    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg" podUID="03d16436-58d3-495e-84d4-38b6d3b55a8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Oct 30 21:50:03.146720 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:03.146690    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1d107b0-5f8m5"]
Oct 30 21:50:03.148175 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:50:03.148138    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda752c8be_55df_4838_b95e_3a643323142b.slice/crio-5d2ed63a8d44821fbc900e7bba2a3ca4f95403ad1c1e47e029a86b5a4e8035f1 WatchSource:0}: Error finding container 5d2ed63a8d44821fbc900e7bba2a3ca4f95403ad1c1e47e029a86b5a4e8035f1: Status 404 returned error can't find the container with id 5d2ed63a8d44821fbc900e7bba2a3ca4f95403ad1c1e47e029a86b5a4e8035f1
Oct 30 21:50:03.500097 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:03.499985    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-ebf644d4-6b5d7b4747-7jqqd" podUID="452e61af-d766-4929-b607-a0ad2332b7cf" containerName="payload-pre-processing" containerID="cri-o://70411834ec3d339c845e14a5338c52f5db69334e8e445a87f2198ac768f05307" gracePeriod=25
Oct 30 21:50:03.801824 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:03.801796    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ebf644d4-6b5d7b4747-7jqqd"
Oct 30 21:50:03.853365 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:03.853335    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/452e61af-d766-4929-b607-a0ad2332b7cf-kube-api-access-55fc7\" (UniqueName: \"kubernetes.io/projected/452e61af-d766-4929-b607-a0ad2332b7cf-kube-api-access-55fc7\") pod \"452e61af-d766-4929-b607-a0ad2332b7cf\" (UID: \"452e61af-d766-4929-b607-a0ad2332b7cf\") "
Oct 30 21:50:03.853548 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:03.853429    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/452e61af-d766-4929-b607-a0ad2332b7cf-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/452e61af-d766-4929-b607-a0ad2332b7cf-plugins-config-volume\") pod \"452e61af-d766-4929-b607-a0ad2332b7cf\" (UID: \"452e61af-d766-4929-b607-a0ad2332b7cf\") "
Oct 30 21:50:03.853815 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:03.853788    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/452e61af-d766-4929-b607-a0ad2332b7cf-plugins-config-volume" pod "452e61af-d766-4929-b607-a0ad2332b7cf" (UID: "452e61af-d766-4929-b607-a0ad2332b7cf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:50:03.855674 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:03.855653    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/452e61af-d766-4929-b607-a0ad2332b7cf-kube-api-access-55fc7" pod "452e61af-d766-4929-b607-a0ad2332b7cf" (UID: "452e61af-d766-4929-b607-a0ad2332b7cf"). InnerVolumeSpecName "kube-api-access-55fc7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:50:03.872037 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:03.872007    4871 patch_prober.go:29] interesting pod/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Oct 30 21:50:03.872141 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:03.872058    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg" podUID="03d16436-58d3-495e-84d4-38b6d3b55a8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Oct 30 21:50:03.952016 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:03.951977    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1d107b0-5f8m5" event={"ID":"a752c8be-55df-4838-b95e-3a643323142b","Type":"ContainerStarted","Data":"d1f0c943ec411b33fb7714dbd3ada0416f82107097690f82a6b23bfea589dea5"}
Oct 30 21:50:03.952482 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:03.952038    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1d107b0-5f8m5" event={"ID":"a752c8be-55df-4838-b95e-3a643323142b","Type":"ContainerStarted","Data":"5d2ed63a8d44821fbc900e7bba2a3ca4f95403ad1c1e47e029a86b5a4e8035f1"}
Oct 30 21:50:03.953559 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:03.953530    4871 generic.go:356] "Generic (PLEG): container finished" podID="452e61af-d766-4929-b607-a0ad2332b7cf" containerID="70411834ec3d339c845e14a5338c52f5db69334e8e445a87f2198ac768f05307" exitCode=0
Oct 30 21:50:03.953704 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:03.953623    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ebf644d4-6b5d7b4747-7jqqd"
Oct 30 21:50:03.954102 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:03.954082    4871 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/452e61af-d766-4929-b607-a0ad2332b7cf-plugins-config-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:03.954200 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:03.954103    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55fc7\" (UniqueName: \"kubernetes.io/projected/452e61af-d766-4929-b607-a0ad2332b7cf-kube-api-access-55fc7\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:03.961300 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:03.961266    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ebf644d4-6b5d7b4747-7jqqd" event={"ID":"452e61af-d766-4929-b607-a0ad2332b7cf","Type":"ContainerDied","Data":"70411834ec3d339c845e14a5338c52f5db69334e8e445a87f2198ac768f05307"}
Oct 30 21:50:03.961404 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:03.961328    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ebf644d4-6b5d7b4747-7jqqd" event={"ID":"452e61af-d766-4929-b607-a0ad2332b7cf","Type":"ContainerDied","Data":"72979efb8acd30f55f642fac9b8b840487d81b7913aea47f56a00e4ba4c2fcf1"}
Oct 30 21:50:03.961404 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:03.961358    4871 scope.go:122] "RemoveContainer" containerID="70411834ec3d339c845e14a5338c52f5db69334e8e445a87f2198ac768f05307"
Oct 30 21:50:03.974320 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:03.974296    4871 scope.go:122] "RemoveContainer" containerID="70411834ec3d339c845e14a5338c52f5db69334e8e445a87f2198ac768f05307"
Oct 30 21:50:03.974660 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:50:03.974637    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70411834ec3d339c845e14a5338c52f5db69334e8e445a87f2198ac768f05307\": container with ID starting with 70411834ec3d339c845e14a5338c52f5db69334e8e445a87f2198ac768f05307 not found: ID does not exist" containerID="70411834ec3d339c845e14a5338c52f5db69334e8e445a87f2198ac768f05307"
Oct 30 21:50:03.974707 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:03.974670    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70411834ec3d339c845e14a5338c52f5db69334e8e445a87f2198ac768f05307"} err="failed to get container status \"70411834ec3d339c845e14a5338c52f5db69334e8e445a87f2198ac768f05307\": rpc error: code = NotFound desc = could not find container \"70411834ec3d339c845e14a5338c52f5db69334e8e445a87f2198ac768f05307\": container with ID starting with 70411834ec3d339c845e14a5338c52f5db69334e8e445a87f2198ac768f05307 not found: ID does not exist"
Oct 30 21:50:04.063691 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:04.063596    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1d107b0-5f8m5" podStartSLOduration=2.063583853 podStartE2EDuration="2.063583853s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:50:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:50:04.060834196 +0000 UTC m=+746.238322813" watchObservedRunningTime="2026-10-30 21:50:04.063583853 +0000 UTC m=+746.241072489"
Oct 30 21:50:04.181334 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:04.181297    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ebf644d4-6b5d7b4747-7jqqd"]
Oct 30 21:50:04.186118 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:04.186083    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ebf644d4-6b5d7b4747-7jqqd"]
Oct 30 21:50:04.872101 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:04.872062    4871 patch_prober.go:29] interesting pod/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Oct 30 21:50:04.872309 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:04.872125    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg" podUID="03d16436-58d3-495e-84d4-38b6d3b55a8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Oct 30 21:50:04.961989 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:04.961951    4871 generic.go:356] "Generic (PLEG): container finished" podID="a752c8be-55df-4838-b95e-3a643323142b" containerID="d1f0c943ec411b33fb7714dbd3ada0416f82107097690f82a6b23bfea589dea5" exitCode=0
Oct 30 21:50:04.962455 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:04.962041    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1d107b0-5f8m5" event={"ID":"a752c8be-55df-4838-b95e-3a643323142b","Type":"ContainerDied","Data":"d1f0c943ec411b33fb7714dbd3ada0416f82107097690f82a6b23bfea589dea5"}
Oct 30 21:50:05.250221 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:05.250143    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:05.568538 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:50:05.568453    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:50:05.871292 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:05.871228    4871 patch_prober.go:29] interesting pod/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Oct 30 21:50:05.871464 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:05.871317    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg" podUID="03d16436-58d3-495e-84d4-38b6d3b55a8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Oct 30 21:50:05.959806 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:05.959772    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="452e61af-d766-4929-b607-a0ad2332b7cf" path="/var/lib/kubelet/pods/452e61af-d766-4929-b607-a0ad2332b7cf/volumes"
Oct 30 21:50:06.126293 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:06.126215    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1d107b0-5f8m5"
Oct 30 21:50:06.176147 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:06.176115    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a752c8be-55df-4838-b95e-3a643323142b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a752c8be-55df-4838-b95e-3a643323142b-maas-api-openshift-ca\") pod \"a752c8be-55df-4838-b95e-3a643323142b\" (UID: \"a752c8be-55df-4838-b95e-3a643323142b\") "
Oct 30 21:50:06.176369 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:06.176198    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a752c8be-55df-4838-b95e-3a643323142b-kube-api-access-w8ncc\" (UniqueName: \"kubernetes.io/projected/a752c8be-55df-4838-b95e-3a643323142b-kube-api-access-w8ncc\") pod \"a752c8be-55df-4838-b95e-3a643323142b\" (UID: \"a752c8be-55df-4838-b95e-3a643323142b\") "
Oct 30 21:50:06.176369 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:06.176279    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a752c8be-55df-4838-b95e-3a643323142b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a752c8be-55df-4838-b95e-3a643323142b-maas-api-xks-ca\") pod \"a752c8be-55df-4838-b95e-3a643323142b\" (UID: \"a752c8be-55df-4838-b95e-3a643323142b\") "
Oct 30 21:50:06.176573 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:06.176544    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a752c8be-55df-4838-b95e-3a643323142b-maas-api-openshift-ca" pod "a752c8be-55df-4838-b95e-3a643323142b" (UID: "a752c8be-55df-4838-b95e-3a643323142b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:50:06.178468 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:06.178439    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a752c8be-55df-4838-b95e-3a643323142b-maas-api-xks-ca" pod "a752c8be-55df-4838-b95e-3a643323142b" (UID: "a752c8be-55df-4838-b95e-3a643323142b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:50:06.178582 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:06.178481    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a752c8be-55df-4838-b95e-3a643323142b-kube-api-access-w8ncc" pod "a752c8be-55df-4838-b95e-3a643323142b" (UID: "a752c8be-55df-4838-b95e-3a643323142b"). InnerVolumeSpecName "kube-api-access-w8ncc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:50:06.277862 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:06.277824    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a752c8be-55df-4838-b95e-3a643323142b-maas-api-xks-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:06.277862 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:06.277857    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a752c8be-55df-4838-b95e-3a643323142b-maas-api-openshift-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:06.277862 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:06.277868    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8ncc\" (UniqueName: \"kubernetes.io/projected/a752c8be-55df-4838-b95e-3a643323142b-kube-api-access-w8ncc\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:06.875785 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:06.875753    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"
Oct 30 21:50:06.876059 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:06.876036    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"
Oct 30 21:50:06.876807 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:06.876792    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"
Oct 30 21:50:06.975909 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:06.975875    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-09d8fb9b-7bc65f5f7f-hl4mj"]
Oct 30 21:50:06.977907 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:06.977885    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1d107b0-5f8m5"
Oct 30 21:50:06.978054 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:06.977932    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1d107b0-5f8m5" event={"ID":"a752c8be-55df-4838-b95e-3a643323142b","Type":"ContainerDied","Data":"5d2ed63a8d44821fbc900e7bba2a3ca4f95403ad1c1e47e029a86b5a4e8035f1"}
Oct 30 21:50:06.978054 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:06.977986    4871 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5d2ed63a8d44821fbc900e7bba2a3ca4f95403ad1c1e47e029a86b5a4e8035f1"
Oct 30 21:50:07.077183 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:07.077140    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-09d8fb9b-54956d9467-lncnc"]
Oct 30 21:50:07.356311 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:07.356215    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:07.518475 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:07.518426    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:09.578060 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:09.578013    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:10.770103 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:10.770065    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:11.120897 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:11.120860    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:11.383679 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:11.383583    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:11.579771 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:11.579734    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ebf644d4-rppbl"]
Oct 30 21:50:11.648318 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:11.647378    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ebf644d4-rppbl"]
Oct 30 21:50:11.959424 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:11.959346    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="53f636b4-eeb0-4a65-a391-965a83079670" path="/var/lib/kubelet/pods/53f636b4-eeb0-4a65-a391-965a83079670/volumes"
Oct 30 21:50:11.976522 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:11.976489    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-09d8fb9b-7bc65f5f7f-hl4mj" podUID="34f376f3-c14a-4736-9ce5-770d628e800e" containerName="payload-processing" containerID="cri-o://0a44aaf2e46030c003f7b2e56a0e981a370abd93066dab5b81a1d18463027b2c" gracePeriod=25
Oct 30 21:50:12.078322 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:12.078233    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-09d8fb9b-54956d9467-lncnc" podUID="57ee4e81-90ce-4670-8522-724918f6b037" containerName="payload-pre-processing" containerID="cri-o://a56c467638f119f8e0d44a4ac780b2bf8945a60b97573aa97645c95b9b5cee85" gracePeriod=25
Oct 30 21:50:12.231893 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:12.231858    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-09d8fb9b-7bc65f5f7f-hl4mj"
Oct 30 21:50:12.333650 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:12.333616    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:12.337669 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:12.337641    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-09d8fb9b-54956d9467-lncnc"
Oct 30 21:50:12.342363 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:12.342337    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/34f376f3-c14a-4736-9ce5-770d628e800e-kube-api-access-5t4h9\" (UniqueName: \"kubernetes.io/projected/34f376f3-c14a-4736-9ce5-770d628e800e-kube-api-access-5t4h9\") pod \"34f376f3-c14a-4736-9ce5-770d628e800e\" (UID: \"34f376f3-c14a-4736-9ce5-770d628e800e\") "
Oct 30 21:50:12.342561 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:12.342539    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/34f376f3-c14a-4736-9ce5-770d628e800e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34f376f3-c14a-4736-9ce5-770d628e800e-plugins-config-volume\") pod \"34f376f3-c14a-4736-9ce5-770d628e800e\" (UID: \"34f376f3-c14a-4736-9ce5-770d628e800e\") "
Oct 30 21:50:12.342963 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:12.342936    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34f376f3-c14a-4736-9ce5-770d628e800e-plugins-config-volume" pod "34f376f3-c14a-4736-9ce5-770d628e800e" (UID: "34f376f3-c14a-4736-9ce5-770d628e800e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:50:12.344677 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:12.344652    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34f376f3-c14a-4736-9ce5-770d628e800e-kube-api-access-5t4h9" pod "34f376f3-c14a-4736-9ce5-770d628e800e" (UID: "34f376f3-c14a-4736-9ce5-770d628e800e"). InnerVolumeSpecName "kube-api-access-5t4h9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:50:12.444292 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:12.444226    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/57ee4e81-90ce-4670-8522-724918f6b037-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57ee4e81-90ce-4670-8522-724918f6b037-plugins-config-volume\") pod \"57ee4e81-90ce-4670-8522-724918f6b037\" (UID: \"57ee4e81-90ce-4670-8522-724918f6b037\") "
Oct 30 21:50:12.444469 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:12.444335    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/57ee4e81-90ce-4670-8522-724918f6b037-kube-api-access-29m8k\" (UniqueName: \"kubernetes.io/projected/57ee4e81-90ce-4670-8522-724918f6b037-kube-api-access-29m8k\") pod \"57ee4e81-90ce-4670-8522-724918f6b037\" (UID: \"57ee4e81-90ce-4670-8522-724918f6b037\") "
Oct 30 21:50:12.444568 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:12.444556    4871 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34f376f3-c14a-4736-9ce5-770d628e800e-plugins-config-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:12.444603 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:12.444570    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5t4h9\" (UniqueName: \"kubernetes.io/projected/34f376f3-c14a-4736-9ce5-770d628e800e-kube-api-access-5t4h9\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:12.444640 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:12.444601    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57ee4e81-90ce-4670-8522-724918f6b037-plugins-config-volume" pod "57ee4e81-90ce-4670-8522-724918f6b037" (UID: "57ee4e81-90ce-4670-8522-724918f6b037"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:50:12.446535 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:12.446508    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57ee4e81-90ce-4670-8522-724918f6b037-kube-api-access-29m8k" pod "57ee4e81-90ce-4670-8522-724918f6b037" (UID: "57ee4e81-90ce-4670-8522-724918f6b037"). InnerVolumeSpecName "kube-api-access-29m8k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:50:12.546179 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:12.546092    4871 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57ee4e81-90ce-4670-8522-724918f6b037-plugins-config-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:12.546179 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:12.546125    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-29m8k\" (UniqueName: \"kubernetes.io/projected/57ee4e81-90ce-4670-8522-724918f6b037-kube-api-access-29m8k\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:12.602432 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:12.602397    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:12.765351 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:12.765319    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:12.845799 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:12.845693    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-09d8fb9b-thlrl"]
Oct 30 21:50:12.848513 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:12.848481    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-09d8fb9b-thlrl"]
Oct 30 21:50:13.027295 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.027264    4871 generic.go:356] "Generic (PLEG): container finished" podID="34f376f3-c14a-4736-9ce5-770d628e800e" containerID="0a44aaf2e46030c003f7b2e56a0e981a370abd93066dab5b81a1d18463027b2c" exitCode=0
Oct 30 21:50:13.027793 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.027321    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-09d8fb9b-7bc65f5f7f-hl4mj"
Oct 30 21:50:13.027793 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.027333    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-09d8fb9b-7bc65f5f7f-hl4mj" event={"ID":"34f376f3-c14a-4736-9ce5-770d628e800e","Type":"ContainerDied","Data":"0a44aaf2e46030c003f7b2e56a0e981a370abd93066dab5b81a1d18463027b2c"}
Oct 30 21:50:13.027793 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.027407    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-09d8fb9b-7bc65f5f7f-hl4mj" event={"ID":"34f376f3-c14a-4736-9ce5-770d628e800e","Type":"ContainerDied","Data":"8443e25344819cef90493fcb1f96904d670172501cb64d07907a3572ded12ee1"}
Oct 30 21:50:13.027793 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.027445    4871 scope.go:122] "RemoveContainer" containerID="0a44aaf2e46030c003f7b2e56a0e981a370abd93066dab5b81a1d18463027b2c"
Oct 30 21:50:13.029419 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.029399    4871 generic.go:356] "Generic (PLEG): container finished" podID="57ee4e81-90ce-4670-8522-724918f6b037" containerID="a56c467638f119f8e0d44a4ac780b2bf8945a60b97573aa97645c95b9b5cee85" exitCode=0
Oct 30 21:50:13.029503 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.029450    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-09d8fb9b-54956d9467-lncnc"
Oct 30 21:50:13.029554 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.029487    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-09d8fb9b-54956d9467-lncnc" event={"ID":"57ee4e81-90ce-4670-8522-724918f6b037","Type":"ContainerDied","Data":"a56c467638f119f8e0d44a4ac780b2bf8945a60b97573aa97645c95b9b5cee85"}
Oct 30 21:50:13.029554 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.029541    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-09d8fb9b-54956d9467-lncnc" event={"ID":"57ee4e81-90ce-4670-8522-724918f6b037","Type":"ContainerDied","Data":"30eb04e3d4c9abb65130702092f0e2ff9c0e31fc3b1803e8bf9f44100b7a8051"}
Oct 30 21:50:13.039398 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.039378    4871 scope.go:122] "RemoveContainer" containerID="0a44aaf2e46030c003f7b2e56a0e981a370abd93066dab5b81a1d18463027b2c"
Oct 30 21:50:13.039711 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:50:13.039691    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a44aaf2e46030c003f7b2e56a0e981a370abd93066dab5b81a1d18463027b2c\": container with ID starting with 0a44aaf2e46030c003f7b2e56a0e981a370abd93066dab5b81a1d18463027b2c not found: ID does not exist" containerID="0a44aaf2e46030c003f7b2e56a0e981a370abd93066dab5b81a1d18463027b2c"
Oct 30 21:50:13.039790 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.039721    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a44aaf2e46030c003f7b2e56a0e981a370abd93066dab5b81a1d18463027b2c"} err="failed to get container status \"0a44aaf2e46030c003f7b2e56a0e981a370abd93066dab5b81a1d18463027b2c\": rpc error: code = NotFound desc = could not find container \"0a44aaf2e46030c003f7b2e56a0e981a370abd93066dab5b81a1d18463027b2c\": container with ID starting with 0a44aaf2e46030c003f7b2e56a0e981a370abd93066dab5b81a1d18463027b2c not found: ID does not exist"
Oct 30 21:50:13.039790 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.039744    4871 scope.go:122] "RemoveContainer" containerID="a56c467638f119f8e0d44a4ac780b2bf8945a60b97573aa97645c95b9b5cee85"
Oct 30 21:50:13.050204 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.050186    4871 scope.go:122] "RemoveContainer" containerID="a56c467638f119f8e0d44a4ac780b2bf8945a60b97573aa97645c95b9b5cee85"
Oct 30 21:50:13.050478 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:50:13.050456    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a56c467638f119f8e0d44a4ac780b2bf8945a60b97573aa97645c95b9b5cee85\": container with ID starting with a56c467638f119f8e0d44a4ac780b2bf8945a60b97573aa97645c95b9b5cee85 not found: ID does not exist" containerID="a56c467638f119f8e0d44a4ac780b2bf8945a60b97573aa97645c95b9b5cee85"
Oct 30 21:50:13.050534 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.050488    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a56c467638f119f8e0d44a4ac780b2bf8945a60b97573aa97645c95b9b5cee85"} err="failed to get container status \"a56c467638f119f8e0d44a4ac780b2bf8945a60b97573aa97645c95b9b5cee85\": rpc error: code = NotFound desc = could not find container \"a56c467638f119f8e0d44a4ac780b2bf8945a60b97573aa97645c95b9b5cee85\": container with ID starting with a56c467638f119f8e0d44a4ac780b2bf8945a60b97573aa97645c95b9b5cee85 not found: ID does not exist"
Oct 30 21:50:13.158144 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.158107    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:13.183161 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.183119    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-09d8fb9b-54956d9467-lncnc"]
Oct 30 21:50:13.187977 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.187943    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-09d8fb9b-54956d9467-lncnc"]
Oct 30 21:50:13.298468 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.294592    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-09d8fb9b-7bc65f5f7f-hl4mj"]
Oct 30 21:50:13.305707 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.305674    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-09d8fb9b-7bc65f5f7f-hl4mj"]
Oct 30 21:50:13.444596 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.444498    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:13.468793 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.468753    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-16c49820-z8bwc"]
Oct 30 21:50:13.469311 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.469287    4871 cpu_manager.go:414] "containerMap: removing container" podUID="a752c8be-55df-4838-b95e-3a643323142b" containerName="revoke-keys"
Oct 30 21:50:13.469424 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.469313    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a752c8be-55df-4838-b95e-3a643323142b" containerName="revoke-keys"
Oct 30 21:50:13.469424 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.469349    4871 cpu_manager.go:414] "containerMap: removing container" podUID="34f376f3-c14a-4736-9ce5-770d628e800e" containerName="payload-processing"
Oct 30 21:50:13.469424 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.469357    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="34f376f3-c14a-4736-9ce5-770d628e800e" containerName="payload-processing"
Oct 30 21:50:13.469424 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.469384    4871 cpu_manager.go:414] "containerMap: removing container" podUID="452e61af-d766-4929-b607-a0ad2332b7cf" containerName="payload-pre-processing"
Oct 30 21:50:13.469424 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.469393    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="452e61af-d766-4929-b607-a0ad2332b7cf" containerName="payload-pre-processing"
Oct 30 21:50:13.469424 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.469423    4871 cpu_manager.go:414] "containerMap: removing container" podUID="57ee4e81-90ce-4670-8522-724918f6b037" containerName="payload-pre-processing"
Oct 30 21:50:13.469712 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.469431    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="57ee4e81-90ce-4670-8522-724918f6b037" containerName="payload-pre-processing"
Oct 30 21:50:13.469712 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.469560    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="57ee4e81-90ce-4670-8522-724918f6b037" containerName="payload-pre-processing"
Oct 30 21:50:13.469712 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.469574    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="a752c8be-55df-4838-b95e-3a643323142b" containerName="revoke-keys"
Oct 30 21:50:13.469712 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.469588    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="34f376f3-c14a-4736-9ce5-770d628e800e" containerName="payload-processing"
Oct 30 21:50:13.469712 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.469599    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="452e61af-d766-4929-b607-a0ad2332b7cf" containerName="payload-pre-processing"
Oct 30 21:50:13.472507 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.472486    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-16c49820-z8bwc"
Oct 30 21:50:13.525640 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.525608    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 21:50:13.526576 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.525973    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 21:50:13.532297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.531439    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-16c49820-z8bwc"]
Oct 30 21:50:13.576403 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.576369    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cdcbb401-ed43-47af-9f55-0123c8b19e6e-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-migrate-16c49820-z8bwc\" (UID: \"cdcbb401-ed43-47af-9f55-0123c8b19e6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-16c49820-z8bwc"
Oct 30 21:50:13.576601 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.576507    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jxld\" (UniqueName: \"kubernetes.io/projected/cdcbb401-ed43-47af-9f55-0123c8b19e6e-kube-api-access-9jxld\") pod \"maas-api-revoke-keys-e2e-migrate-16c49820-z8bwc\" (UID: \"cdcbb401-ed43-47af-9f55-0123c8b19e6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-16c49820-z8bwc"
Oct 30 21:50:13.576601 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.576543    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cdcbb401-ed43-47af-9f55-0123c8b19e6e-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-migrate-16c49820-z8bwc\" (UID: \"cdcbb401-ed43-47af-9f55-0123c8b19e6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-16c49820-z8bwc"
Oct 30 21:50:13.580474 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.580445    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:50:13.580604 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.580484    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mlvtf\""
Oct 30 21:50:13.677518 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.677475    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cdcbb401-ed43-47af-9f55-0123c8b19e6e-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-migrate-16c49820-z8bwc\" (UID: \"cdcbb401-ed43-47af-9f55-0123c8b19e6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-16c49820-z8bwc"
Oct 30 21:50:13.677684 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.677576    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jxld\" (UniqueName: \"kubernetes.io/projected/cdcbb401-ed43-47af-9f55-0123c8b19e6e-kube-api-access-9jxld\") pod \"maas-api-revoke-keys-e2e-migrate-16c49820-z8bwc\" (UID: \"cdcbb401-ed43-47af-9f55-0123c8b19e6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-16c49820-z8bwc"
Oct 30 21:50:13.677684 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.677601    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cdcbb401-ed43-47af-9f55-0123c8b19e6e-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-migrate-16c49820-z8bwc\" (UID: \"cdcbb401-ed43-47af-9f55-0123c8b19e6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-16c49820-z8bwc"
Oct 30 21:50:13.678311 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.678284    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cdcbb401-ed43-47af-9f55-0123c8b19e6e-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-migrate-16c49820-z8bwc\" (UID: \"cdcbb401-ed43-47af-9f55-0123c8b19e6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-16c49820-z8bwc"
Oct 30 21:50:13.679907 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.679889    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cdcbb401-ed43-47af-9f55-0123c8b19e6e-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-migrate-16c49820-z8bwc\" (UID: \"cdcbb401-ed43-47af-9f55-0123c8b19e6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-16c49820-z8bwc"
Oct 30 21:50:13.736132 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.736059    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jxld\" (UniqueName: \"kubernetes.io/projected/cdcbb401-ed43-47af-9f55-0123c8b19e6e-kube-api-access-9jxld\") pod \"maas-api-revoke-keys-e2e-migrate-16c49820-z8bwc\" (UID: \"cdcbb401-ed43-47af-9f55-0123c8b19e6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-16c49820-z8bwc"
Oct 30 21:50:13.793391 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.793354    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-16c49820-z8bwc"
Oct 30 21:50:13.959632 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.959599    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="34f376f3-c14a-4736-9ce5-770d628e800e" path="/var/lib/kubelet/pods/34f376f3-c14a-4736-9ce5-770d628e800e/volumes"
Oct 30 21:50:13.960021 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.960008    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="57ee4e81-90ce-4670-8522-724918f6b037" path="/var/lib/kubelet/pods/57ee4e81-90ce-4670-8522-724918f6b037/volumes"
Oct 30 21:50:13.960398 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:13.960385    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a9dfd1e8-cfd1-414a-92d5-1241a669d360" path="/var/lib/kubelet/pods/a9dfd1e8-cfd1-414a-92d5-1241a669d360/volumes"
Oct 30 21:50:14.170870 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:14.170837    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-16c49820-z8bwc"]
Oct 30 21:50:14.171491 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:50:14.171462    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdcbb401_ed43_47af_9f55_0123c8b19e6e.slice/crio-59abdad72fd102393b89a51963d0e86576a850abf82bbdead357f34b7bed9336 WatchSource:0}: Error finding container 59abdad72fd102393b89a51963d0e86576a850abf82bbdead357f34b7bed9336: Status 404 returned error can't find the container with id 59abdad72fd102393b89a51963d0e86576a850abf82bbdead357f34b7bed9336
Oct 30 21:50:14.586463 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:14.586423    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:14.832352 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:14.831873    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:15.048743 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:15.048688    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-16c49820-z8bwc" event={"ID":"cdcbb401-ed43-47af-9f55-0123c8b19e6e","Type":"ContainerStarted","Data":"b06975fb9e946bc67a19fd11847216ea377136fefd84ee0c3c7512c7630aed03"}
Oct 30 21:50:15.049017 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:15.048984    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-16c49820-z8bwc" event={"ID":"cdcbb401-ed43-47af-9f55-0123c8b19e6e","Type":"ContainerStarted","Data":"59abdad72fd102393b89a51963d0e86576a850abf82bbdead357f34b7bed9336"}
Oct 30 21:50:15.160717 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:15.160659    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-16c49820-z8bwc" podStartSLOduration=2.160643162 podStartE2EDuration="2.160643162s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:50:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:50:15.158554957 +0000 UTC m=+757.336043575" watchObservedRunningTime="2026-10-30 21:50:15.160643162 +0000 UTC m=+757.338131779"
Oct 30 21:50:16.058114 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:16.058077    4871 generic.go:356] "Generic (PLEG): container finished" podID="cdcbb401-ed43-47af-9f55-0123c8b19e6e" containerID="b06975fb9e946bc67a19fd11847216ea377136fefd84ee0c3c7512c7630aed03" exitCode=0
Oct 30 21:50:16.058564 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:16.058161    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-16c49820-z8bwc" event={"ID":"cdcbb401-ed43-47af-9f55-0123c8b19e6e","Type":"ContainerDied","Data":"b06975fb9e946bc67a19fd11847216ea377136fefd84ee0c3c7512c7630aed03"}
Oct 30 21:50:17.225113 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:17.225086    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-16c49820-z8bwc"
Oct 30 21:50:17.314580 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:17.314542    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cdcbb401-ed43-47af-9f55-0123c8b19e6e-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cdcbb401-ed43-47af-9f55-0123c8b19e6e-maas-api-openshift-ca\") pod \"cdcbb401-ed43-47af-9f55-0123c8b19e6e\" (UID: \"cdcbb401-ed43-47af-9f55-0123c8b19e6e\") "
Oct 30 21:50:17.314785 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:17.314697    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cdcbb401-ed43-47af-9f55-0123c8b19e6e-kube-api-access-9jxld\" (UniqueName: \"kubernetes.io/projected/cdcbb401-ed43-47af-9f55-0123c8b19e6e-kube-api-access-9jxld\") pod \"cdcbb401-ed43-47af-9f55-0123c8b19e6e\" (UID: \"cdcbb401-ed43-47af-9f55-0123c8b19e6e\") "
Oct 30 21:50:17.314785 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:17.314754    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cdcbb401-ed43-47af-9f55-0123c8b19e6e-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cdcbb401-ed43-47af-9f55-0123c8b19e6e-maas-api-xks-ca\") pod \"cdcbb401-ed43-47af-9f55-0123c8b19e6e\" (UID: \"cdcbb401-ed43-47af-9f55-0123c8b19e6e\") "
Oct 30 21:50:17.315006 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:17.314979    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdcbb401-ed43-47af-9f55-0123c8b19e6e-maas-api-openshift-ca" pod "cdcbb401-ed43-47af-9f55-0123c8b19e6e" (UID: "cdcbb401-ed43-47af-9f55-0123c8b19e6e"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:50:17.316862 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:17.316801    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdcbb401-ed43-47af-9f55-0123c8b19e6e-kube-api-access-9jxld" pod "cdcbb401-ed43-47af-9f55-0123c8b19e6e" (UID: "cdcbb401-ed43-47af-9f55-0123c8b19e6e"). InnerVolumeSpecName "kube-api-access-9jxld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:50:17.316955 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:17.316861    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdcbb401-ed43-47af-9f55-0123c8b19e6e-maas-api-xks-ca" pod "cdcbb401-ed43-47af-9f55-0123c8b19e6e" (UID: "cdcbb401-ed43-47af-9f55-0123c8b19e6e"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:50:17.416508 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:17.416474    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jxld\" (UniqueName: \"kubernetes.io/projected/cdcbb401-ed43-47af-9f55-0123c8b19e6e-kube-api-access-9jxld\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:17.416508 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:17.416508    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cdcbb401-ed43-47af-9f55-0123c8b19e6e-maas-api-xks-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:17.416719 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:17.416519    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cdcbb401-ed43-47af-9f55-0123c8b19e6e-maas-api-openshift-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:17.449081 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:17.449046    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:18.074340 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:18.074312    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-16c49820-z8bwc"
Oct 30 21:50:18.074532 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:18.074312    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-16c49820-z8bwc" event={"ID":"cdcbb401-ed43-47af-9f55-0123c8b19e6e","Type":"ContainerDied","Data":"59abdad72fd102393b89a51963d0e86576a850abf82bbdead357f34b7bed9336"}
Oct 30 21:50:18.074532 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:18.074440    4871 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="59abdad72fd102393b89a51963d0e86576a850abf82bbdead357f34b7bed9336"
Oct 30 21:50:19.569553 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:19.569508    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ffe7a1cf-kl9xs"]
Oct 30 21:50:19.570200 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:19.570174    4871 cpu_manager.go:414] "containerMap: removing container" podUID="cdcbb401-ed43-47af-9f55-0123c8b19e6e" containerName="revoke-keys"
Oct 30 21:50:19.570309 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:19.570203    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cdcbb401-ed43-47af-9f55-0123c8b19e6e" containerName="revoke-keys"
Oct 30 21:50:19.570388 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:19.570373    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="cdcbb401-ed43-47af-9f55-0123c8b19e6e" containerName="revoke-keys"
Oct 30 21:50:19.572686 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:19.572661    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ffe7a1cf-kl9xs"
Oct 30 21:50:19.622027 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:19.621986    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:50:19.639511 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:19.639473    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/df948643-feaf-4ed8-b5e9-9e2e5e075e83-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-derive-ffe7a1cf-kl9xs\" (UID: \"df948643-feaf-4ed8-b5e9-9e2e5e075e83\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ffe7a1cf-kl9xs"
Oct 30 21:50:19.639703 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:19.639525    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/df948643-feaf-4ed8-b5e9-9e2e5e075e83-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-derive-ffe7a1cf-kl9xs\" (UID: \"df948643-feaf-4ed8-b5e9-9e2e5e075e83\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ffe7a1cf-kl9xs"
Oct 30 21:50:19.639817 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:19.639777    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bqfg\" (UniqueName: \"kubernetes.io/projected/df948643-feaf-4ed8-b5e9-9e2e5e075e83-kube-api-access-7bqfg\") pod \"maas-api-revoke-keys-e2e-derive-ffe7a1cf-kl9xs\" (UID: \"df948643-feaf-4ed8-b5e9-9e2e5e075e83\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ffe7a1cf-kl9xs"
Oct 30 21:50:19.668710 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:19.668670    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mlvtf\""
Oct 30 21:50:19.668904 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:19.668756    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 21:50:19.668904 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:19.668771    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 21:50:19.671929 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:19.671902    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ffe7a1cf-kl9xs"]
Oct 30 21:50:19.741004 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:19.740972    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bqfg\" (UniqueName: \"kubernetes.io/projected/df948643-feaf-4ed8-b5e9-9e2e5e075e83-kube-api-access-7bqfg\") pod \"maas-api-revoke-keys-e2e-derive-ffe7a1cf-kl9xs\" (UID: \"df948643-feaf-4ed8-b5e9-9e2e5e075e83\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ffe7a1cf-kl9xs"
Oct 30 21:50:19.741191 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:19.741046    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/df948643-feaf-4ed8-b5e9-9e2e5e075e83-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-derive-ffe7a1cf-kl9xs\" (UID: \"df948643-feaf-4ed8-b5e9-9e2e5e075e83\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ffe7a1cf-kl9xs"
Oct 30 21:50:19.741191 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:19.741087    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/df948643-feaf-4ed8-b5e9-9e2e5e075e83-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-derive-ffe7a1cf-kl9xs\" (UID: \"df948643-feaf-4ed8-b5e9-9e2e5e075e83\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ffe7a1cf-kl9xs"
Oct 30 21:50:19.741773 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:19.741743    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/df948643-feaf-4ed8-b5e9-9e2e5e075e83-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-derive-ffe7a1cf-kl9xs\" (UID: \"df948643-feaf-4ed8-b5e9-9e2e5e075e83\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ffe7a1cf-kl9xs"
Oct 30 21:50:19.743519 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:19.743499    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/df948643-feaf-4ed8-b5e9-9e2e5e075e83-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-derive-ffe7a1cf-kl9xs\" (UID: \"df948643-feaf-4ed8-b5e9-9e2e5e075e83\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ffe7a1cf-kl9xs"
Oct 30 21:50:19.799074 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:19.799039    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bqfg\" (UniqueName: \"kubernetes.io/projected/df948643-feaf-4ed8-b5e9-9e2e5e075e83-kube-api-access-7bqfg\") pod \"maas-api-revoke-keys-e2e-derive-ffe7a1cf-kl9xs\" (UID: \"df948643-feaf-4ed8-b5e9-9e2e5e075e83\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ffe7a1cf-kl9xs"
Oct 30 21:50:19.885763 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:19.885727    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ffe7a1cf-kl9xs"
Oct 30 21:50:20.254610 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:20.254576    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ffe7a1cf-kl9xs"]
Oct 30 21:50:20.255874 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:50:20.255844    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf948643_feaf_4ed8_b5e9_9e2e5e075e83.slice/crio-47420a1a2fd66032d8a5287979f7c71c0d17583eadedaa4520c1cfc948bf2aba WatchSource:0}: Error finding container 47420a1a2fd66032d8a5287979f7c71c0d17583eadedaa4520c1cfc948bf2aba: Status 404 returned error can't find the container with id 47420a1a2fd66032d8a5287979f7c71c0d17583eadedaa4520c1cfc948bf2aba
Oct 30 21:50:21.097030 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:21.096992    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ffe7a1cf-kl9xs" event={"ID":"df948643-feaf-4ed8-b5e9-9e2e5e075e83","Type":"ContainerStarted","Data":"d1ad1927ef91e1f7f85c3da00f156f8fd73cdfb68a1cbff9f4c71fad02edd90f"}
Oct 30 21:50:21.097492 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:21.097051    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ffe7a1cf-kl9xs" event={"ID":"df948643-feaf-4ed8-b5e9-9e2e5e075e83","Type":"ContainerStarted","Data":"47420a1a2fd66032d8a5287979f7c71c0d17583eadedaa4520c1cfc948bf2aba"}
Oct 30 21:50:21.208906 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:21.208833    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ffe7a1cf-kl9xs" podStartSLOduration=2.208809549 podStartE2EDuration="2.208809549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:50:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:50:21.206095364 +0000 UTC m=+763.383583984" watchObservedRunningTime="2026-10-30 21:50:21.208809549 +0000 UTC m=+763.386298167"
Oct 30 21:50:22.106252 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:22.106203    4871 generic.go:356] "Generic (PLEG): container finished" podID="df948643-feaf-4ed8-b5e9-9e2e5e075e83" containerID="d1ad1927ef91e1f7f85c3da00f156f8fd73cdfb68a1cbff9f4c71fad02edd90f" exitCode=0
Oct 30 21:50:22.106694 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:22.106289    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ffe7a1cf-kl9xs" event={"ID":"df948643-feaf-4ed8-b5e9-9e2e5e075e83","Type":"ContainerDied","Data":"d1ad1927ef91e1f7f85c3da00f156f8fd73cdfb68a1cbff9f4c71fad02edd90f"}
Oct 30 21:50:22.341218 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:22.341181    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:23.280820 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:23.280796    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ffe7a1cf-kl9xs"
Oct 30 21:50:23.380259 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:23.380161    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/df948643-feaf-4ed8-b5e9-9e2e5e075e83-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/df948643-feaf-4ed8-b5e9-9e2e5e075e83-maas-api-openshift-ca\") pod \"df948643-feaf-4ed8-b5e9-9e2e5e075e83\" (UID: \"df948643-feaf-4ed8-b5e9-9e2e5e075e83\") "
Oct 30 21:50:23.380259 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:23.380220    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/df948643-feaf-4ed8-b5e9-9e2e5e075e83-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/df948643-feaf-4ed8-b5e9-9e2e5e075e83-maas-api-xks-ca\") pod \"df948643-feaf-4ed8-b5e9-9e2e5e075e83\" (UID: \"df948643-feaf-4ed8-b5e9-9e2e5e075e83\") "
Oct 30 21:50:23.380504 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:23.380330    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/df948643-feaf-4ed8-b5e9-9e2e5e075e83-kube-api-access-7bqfg\" (UniqueName: \"kubernetes.io/projected/df948643-feaf-4ed8-b5e9-9e2e5e075e83-kube-api-access-7bqfg\") pod \"df948643-feaf-4ed8-b5e9-9e2e5e075e83\" (UID: \"df948643-feaf-4ed8-b5e9-9e2e5e075e83\") "
Oct 30 21:50:23.380567 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:23.380541    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df948643-feaf-4ed8-b5e9-9e2e5e075e83-maas-api-openshift-ca" pod "df948643-feaf-4ed8-b5e9-9e2e5e075e83" (UID: "df948643-feaf-4ed8-b5e9-9e2e5e075e83"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:50:23.382405 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:23.382372    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df948643-feaf-4ed8-b5e9-9e2e5e075e83-maas-api-xks-ca" pod "df948643-feaf-4ed8-b5e9-9e2e5e075e83" (UID: "df948643-feaf-4ed8-b5e9-9e2e5e075e83"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:50:23.382525 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:23.382432    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df948643-feaf-4ed8-b5e9-9e2e5e075e83-kube-api-access-7bqfg" pod "df948643-feaf-4ed8-b5e9-9e2e5e075e83" (UID: "df948643-feaf-4ed8-b5e9-9e2e5e075e83"). InnerVolumeSpecName "kube-api-access-7bqfg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:50:23.481646 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:23.481610    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7bqfg\" (UniqueName: \"kubernetes.io/projected/df948643-feaf-4ed8-b5e9-9e2e5e075e83-kube-api-access-7bqfg\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:23.481646 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:23.481641    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/df948643-feaf-4ed8-b5e9-9e2e5e075e83-maas-api-openshift-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:23.481646 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:23.481652    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/df948643-feaf-4ed8-b5e9-9e2e5e075e83-maas-api-xks-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:24.005946 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:24.005909    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:24.122124 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:24.122092    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ffe7a1cf-kl9xs"
Oct 30 21:50:24.122124 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:24.122113    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ffe7a1cf-kl9xs" event={"ID":"df948643-feaf-4ed8-b5e9-9e2e5e075e83","Type":"ContainerDied","Data":"47420a1a2fd66032d8a5287979f7c71c0d17583eadedaa4520c1cfc948bf2aba"}
Oct 30 21:50:24.122432 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:24.122163    4871 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="47420a1a2fd66032d8a5287979f7c71c0d17583eadedaa4520c1cfc948bf2aba"
Oct 30 21:50:24.280103 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:24.280010    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:24.669773 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:24.669733    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:24.835390 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:24.835356    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:24.995272 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:24.995166    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:24.996825 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:24.996804    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-16c49820-84664c54c-2592k"]
Oct 30 21:50:25.104574 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:25.104544    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-16c49820-5654f658d7-v744c"]
Oct 30 21:50:25.230468 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:25.230413    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:25.398774 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:25.398731    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:25.564067 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:25.564024    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:27.056909 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:27.056876    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:29.522262 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:29.519632    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:29.718389 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:29.718355    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:29.881620 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:29.881585    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:29.998057 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:29.998007    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-16c49820-84664c54c-2592k" podUID="72192587-c584-4ed8-b949-a72df97551d9" containerName="payload-processing" containerID="cri-o://2717cc0c635b383dcd947d57ba11a246f5b39bf9afc51a5162e44318aae1090d" gracePeriod=25
Oct 30 21:50:30.111548 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:30.111318    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-16c49820-5654f658d7-v744c" podUID="b5ce1b1a-89d4-4336-aee6-f71b2f8a0405" containerName="payload-pre-processing" containerID="cri-o://bf16263f0cd06627ce28336b8c6c40cee7594da2b926be22b2964da22d88f8da" gracePeriod=25
Oct 30 21:50:30.163059 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:30.163026    4871 generic.go:356] "Generic (PLEG): container finished" podID="72192587-c584-4ed8-b949-a72df97551d9" containerID="2717cc0c635b383dcd947d57ba11a246f5b39bf9afc51a5162e44318aae1090d" exitCode=0
Oct 30 21:50:30.163207 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:30.163084    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-16c49820-84664c54c-2592k" event={"ID":"72192587-c584-4ed8-b949-a72df97551d9","Type":"ContainerDied","Data":"2717cc0c635b383dcd947d57ba11a246f5b39bf9afc51a5162e44318aae1090d"}
Oct 30 21:50:30.364932 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:30.364905    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-16c49820-84664c54c-2592k"
Oct 30 21:50:30.454689 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:30.454663    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-16c49820-5654f658d7-v744c"
Oct 30 21:50:30.471746 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:30.471711    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/72192587-c584-4ed8-b949-a72df97551d9-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72192587-c584-4ed8-b949-a72df97551d9-plugins-config-volume\") pod \"72192587-c584-4ed8-b949-a72df97551d9\" (UID: \"72192587-c584-4ed8-b949-a72df97551d9\") "
Oct 30 21:50:30.471915 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:30.471778    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/72192587-c584-4ed8-b949-a72df97551d9-kube-api-access-gdl89\" (UniqueName: \"kubernetes.io/projected/72192587-c584-4ed8-b949-a72df97551d9-kube-api-access-gdl89\") pod \"72192587-c584-4ed8-b949-a72df97551d9\" (UID: \"72192587-c584-4ed8-b949-a72df97551d9\") "
Oct 30 21:50:30.472220 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:30.472194    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72192587-c584-4ed8-b949-a72df97551d9-plugins-config-volume" pod "72192587-c584-4ed8-b949-a72df97551d9" (UID: "72192587-c584-4ed8-b949-a72df97551d9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:50:30.474062 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:30.474038    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72192587-c584-4ed8-b949-a72df97551d9-kube-api-access-gdl89" pod "72192587-c584-4ed8-b949-a72df97551d9" (UID: "72192587-c584-4ed8-b949-a72df97551d9"). InnerVolumeSpecName "kube-api-access-gdl89". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:50:30.573978 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:30.573532    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b5ce1b1a-89d4-4336-aee6-f71b2f8a0405-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5ce1b1a-89d4-4336-aee6-f71b2f8a0405-plugins-config-volume\") pod \"b5ce1b1a-89d4-4336-aee6-f71b2f8a0405\" (UID: \"b5ce1b1a-89d4-4336-aee6-f71b2f8a0405\") "
Oct 30 21:50:30.573978 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:30.573601    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b5ce1b1a-89d4-4336-aee6-f71b2f8a0405-kube-api-access-j4fk7\" (UniqueName: \"kubernetes.io/projected/b5ce1b1a-89d4-4336-aee6-f71b2f8a0405-kube-api-access-j4fk7\") pod \"b5ce1b1a-89d4-4336-aee6-f71b2f8a0405\" (UID: \"b5ce1b1a-89d4-4336-aee6-f71b2f8a0405\") "
Oct 30 21:50:30.573978 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:30.573922    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5ce1b1a-89d4-4336-aee6-f71b2f8a0405-plugins-config-volume" pod "b5ce1b1a-89d4-4336-aee6-f71b2f8a0405" (UID: "b5ce1b1a-89d4-4336-aee6-f71b2f8a0405"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:50:30.573978 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:30.573947    4871 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72192587-c584-4ed8-b949-a72df97551d9-plugins-config-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:30.573978 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:30.573964    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdl89\" (UniqueName: \"kubernetes.io/projected/72192587-c584-4ed8-b949-a72df97551d9-kube-api-access-gdl89\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:30.575701 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:30.575672    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5ce1b1a-89d4-4336-aee6-f71b2f8a0405-kube-api-access-j4fk7" pod "b5ce1b1a-89d4-4336-aee6-f71b2f8a0405" (UID: "b5ce1b1a-89d4-4336-aee6-f71b2f8a0405"). InnerVolumeSpecName "kube-api-access-j4fk7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:50:30.674699 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:30.674667    4871 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5ce1b1a-89d4-4336-aee6-f71b2f8a0405-plugins-config-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:30.674699 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:30.674696    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4fk7\" (UniqueName: \"kubernetes.io/projected/b5ce1b1a-89d4-4336-aee6-f71b2f8a0405-kube-api-access-j4fk7\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:31.171283 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:31.171222    4871 generic.go:356] "Generic (PLEG): container finished" podID="b5ce1b1a-89d4-4336-aee6-f71b2f8a0405" containerID="bf16263f0cd06627ce28336b8c6c40cee7594da2b926be22b2964da22d88f8da" exitCode=0
Oct 30 21:50:31.171435 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:31.171417    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-16c49820-5654f658d7-v744c" event={"ID":"b5ce1b1a-89d4-4336-aee6-f71b2f8a0405","Type":"ContainerDied","Data":"bf16263f0cd06627ce28336b8c6c40cee7594da2b926be22b2964da22d88f8da"}
Oct 30 21:50:31.171495 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:31.171460    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-16c49820-5654f658d7-v744c" event={"ID":"b5ce1b1a-89d4-4336-aee6-f71b2f8a0405","Type":"ContainerDied","Data":"e1ed970c8573887c8d8f6f43d659fb99c3b366fb0c1f9f42b2846aa38bf083d6"}
Oct 30 21:50:31.171495 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:31.171482    4871 scope.go:122] "RemoveContainer" containerID="bf16263f0cd06627ce28336b8c6c40cee7594da2b926be22b2964da22d88f8da"
Oct 30 21:50:31.171602 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:31.171591    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-16c49820-5654f658d7-v744c"
Oct 30 21:50:31.174133 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:31.174106    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-16c49820-84664c54c-2592k" event={"ID":"72192587-c584-4ed8-b949-a72df97551d9","Type":"ContainerDied","Data":"3da9ee38dcd0354b5fc6b6d489691eac6fed8b1859e2093d0f33ceac8325f869"}
Oct 30 21:50:31.174282 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:31.174200    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-16c49820-84664c54c-2592k"
Oct 30 21:50:31.184608 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:31.184586    4871 scope.go:122] "RemoveContainer" containerID="bf16263f0cd06627ce28336b8c6c40cee7594da2b926be22b2964da22d88f8da"
Oct 30 21:50:31.185174 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:50:31.185136    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf16263f0cd06627ce28336b8c6c40cee7594da2b926be22b2964da22d88f8da\": container with ID starting with bf16263f0cd06627ce28336b8c6c40cee7594da2b926be22b2964da22d88f8da not found: ID does not exist" containerID="bf16263f0cd06627ce28336b8c6c40cee7594da2b926be22b2964da22d88f8da"
Oct 30 21:50:31.185269 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:31.185172    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf16263f0cd06627ce28336b8c6c40cee7594da2b926be22b2964da22d88f8da"} err="failed to get container status \"bf16263f0cd06627ce28336b8c6c40cee7594da2b926be22b2964da22d88f8da\": rpc error: code = NotFound desc = could not find container \"bf16263f0cd06627ce28336b8c6c40cee7594da2b926be22b2964da22d88f8da\": container with ID starting with bf16263f0cd06627ce28336b8c6c40cee7594da2b926be22b2964da22d88f8da not found: ID does not exist"
Oct 30 21:50:31.185269 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:31.185217    4871 scope.go:122] "RemoveContainer" containerID="2717cc0c635b383dcd947d57ba11a246f5b39bf9afc51a5162e44318aae1090d"
Oct 30 21:50:31.277554 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:31.277513    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-ffe7a1cf-6976c8c597-t5ncj"]
Oct 30 21:50:31.346699 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:31.346552    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-16c49820-84664c54c-2592k"]
Oct 30 21:50:31.361921 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:31.361888    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-16c49820-84664c54c-2592k"]
Oct 30 21:50:31.441279 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:31.440848    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:31.471688 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:31.471645    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-16c49820-5654f658d7-v744c"]
Oct 30 21:50:31.480087 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:31.480054    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-16c49820-5654f658d7-v744c"]
Oct 30 21:50:31.659148 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:31.659030    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:31.827274 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:31.821591    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:31.966453 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:31.966376    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="72192587-c584-4ed8-b949-a72df97551d9" path="/var/lib/kubelet/pods/72192587-c584-4ed8-b949-a72df97551d9/volumes"
Oct 30 21:50:31.967100 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:31.967069    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b5ce1b1a-89d4-4336-aee6-f71b2f8a0405" path="/var/lib/kubelet/pods/b5ce1b1a-89d4-4336-aee6-f71b2f8a0405/volumes"
Oct 30 21:50:32.058930 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:32.058892    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:32.283122 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:32.283049    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:32.951002 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:32.950947    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:33.122972 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:33.122937    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:33.273886 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:33.273802    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:33.411046 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:33.411012    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1d107b0-5f8m5"]
Oct 30 21:50:33.457586 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:33.457546    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1d107b0-5f8m5"]
Oct 30 21:50:33.493041 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:33.492988    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"]
Oct 30 21:50:33.493801 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:33.493764    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw" podUID="d11e9475-1d27-44b3-b433-13a5cc49a13a" containerName="istio-proxy" containerID="cri-o://bee8fa33b605bcadb1c43011eb60ffe6a41f4c9d75be42d381c5721e9f02c800" gracePeriod=30
Oct 30 21:50:33.533693 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:33.533611    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:33.959743 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:33.959705    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a752c8be-55df-4838-b95e-3a643323142b" path="/var/lib/kubelet/pods/a752c8be-55df-4838-b95e-3a643323142b/volumes"
Oct 30 21:50:34.412193 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:34.412161    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:35.053731 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:35.053689    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:35.260295 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:35.260226    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:35.409312 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:35.409282    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:35.455281 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:35.455065    4871 patch_prober.go:29] interesting pod/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": read tcp 10.133.0.2:33148->10.133.0.57:15021: read: connection reset by peer" start-of-body=
Oct 30 21:50:35.455281 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:35.455139    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn" podUID="e626a30e-ff4e-4e87-9f50-000865cfe6c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": read tcp 10.133.0.2:33148->10.133.0.57:15021: read: connection reset by peer"
Oct 30 21:50:35.501865 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:35.501824    4871 patch_prober.go:29] interesting pod/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:50:35.502020 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:35.501893    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw" podUID="d11e9475-1d27-44b3-b433-13a5cc49a13a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:50:35.557012 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:35.556975    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:35.571143 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:50:35.571107    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:50:35.652800 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:35.652760    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-9a5623-6fbf554f99-kwmhs"]
Oct 30 21:50:35.653260 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:35.653232    4871 cpu_manager.go:414] "containerMap: removing container" podUID="72192587-c584-4ed8-b949-a72df97551d9" containerName="payload-processing"
Oct 30 21:50:35.653312 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:35.653261    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="72192587-c584-4ed8-b949-a72df97551d9" containerName="payload-processing"
Oct 30 21:50:35.653312 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:35.653283    4871 cpu_manager.go:414] "containerMap: removing container" podUID="df948643-feaf-4ed8-b5e9-9e2e5e075e83" containerName="revoke-keys"
Oct 30 21:50:35.653312 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:35.653288    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="df948643-feaf-4ed8-b5e9-9e2e5e075e83" containerName="revoke-keys"
Oct 30 21:50:35.653312 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:35.653297    4871 cpu_manager.go:414] "containerMap: removing container" podUID="b5ce1b1a-89d4-4336-aee6-f71b2f8a0405" containerName="payload-pre-processing"
Oct 30 21:50:35.653312 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:35.653302    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b5ce1b1a-89d4-4336-aee6-f71b2f8a0405" containerName="payload-pre-processing"
Oct 30 21:50:35.653483 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:35.653375    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="72192587-c584-4ed8-b949-a72df97551d9" containerName="payload-processing"
Oct 30 21:50:35.653483 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:35.653385    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="df948643-feaf-4ed8-b5e9-9e2e5e075e83" containerName="revoke-keys"
Oct 30 21:50:35.653483 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:35.653393    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="b5ce1b1a-89d4-4336-aee6-f71b2f8a0405" containerName="payload-pre-processing"
Oct 30 21:50:35.655935 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:35.655916    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-9a5623-6fbf554f99-kwmhs"
Oct 30 21:50:35.714097 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:35.714056    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-9a5623-6fbf554f99-kwmhs"]
Oct 30 21:50:35.732921 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:35.732885    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4rnc\" (UniqueName: \"kubernetes.io/projected/16f9970d-5785-4708-a03a-34fc692d3d2a-kube-api-access-c4rnc\") pod \"payload-pre-processing-e2e-worker-w0-9a5623-6fbf554f99-kwmhs\" (UID: \"16f9970d-5785-4708-a03a-34fc692d3d2a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-9a5623-6fbf554f99-kwmhs"
Oct 30 21:50:35.733129 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:35.733003    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16f9970d-5785-4708-a03a-34fc692d3d2a-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-9a5623-6fbf554f99-kwmhs\" (UID: \"16f9970d-5785-4708-a03a-34fc692d3d2a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-9a5623-6fbf554f99-kwmhs"
Oct 30 21:50:35.757277 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:35.757206    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-9a5623-dockercfg-q8z4l\""
Oct 30 21:50:35.757277 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:35.757206    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-9a5623\""
Oct 30 21:50:35.810273 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:35.810224    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-9a5623-597c949cc5-xjcgz"]
Oct 30 21:50:35.812978 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:35.812956    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-9a5623-597c949cc5-xjcgz"
Oct 30 21:50:35.833434 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:35.833400    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qcjr\" (UniqueName: \"kubernetes.io/projected/80d68d4f-bef1-438c-a194-45506f62291c-kube-api-access-4qcjr\") pod \"payload-processing-e2e-worker-w0-9a5623-597c949cc5-xjcgz\" (UID: \"80d68d4f-bef1-438c-a194-45506f62291c\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-9a5623-597c949cc5-xjcgz"
Oct 30 21:50:35.833592 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:35.833442    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4rnc\" (UniqueName: \"kubernetes.io/projected/16f9970d-5785-4708-a03a-34fc692d3d2a-kube-api-access-c4rnc\") pod \"payload-pre-processing-e2e-worker-w0-9a5623-6fbf554f99-kwmhs\" (UID: \"16f9970d-5785-4708-a03a-34fc692d3d2a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-9a5623-6fbf554f99-kwmhs"
Oct 30 21:50:35.833592 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:35.833512    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80d68d4f-bef1-438c-a194-45506f62291c-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-9a5623-597c949cc5-xjcgz\" (UID: \"80d68d4f-bef1-438c-a194-45506f62291c\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-9a5623-597c949cc5-xjcgz"
Oct 30 21:50:35.833685 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:35.833620    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16f9970d-5785-4708-a03a-34fc692d3d2a-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-9a5623-6fbf554f99-kwmhs\" (UID: \"16f9970d-5785-4708-a03a-34fc692d3d2a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-9a5623-6fbf554f99-kwmhs"
Oct 30 21:50:35.834325 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:35.834306    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16f9970d-5785-4708-a03a-34fc692d3d2a-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-9a5623-6fbf554f99-kwmhs\" (UID: \"16f9970d-5785-4708-a03a-34fc692d3d2a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-9a5623-6fbf554f99-kwmhs"
Oct 30 21:50:35.870632 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:35.870599    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-9a5623-597c949cc5-xjcgz"]
Oct 30 21:50:35.891944 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:35.891908    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4rnc\" (UniqueName: \"kubernetes.io/projected/16f9970d-5785-4708-a03a-34fc692d3d2a-kube-api-access-c4rnc\") pod \"payload-pre-processing-e2e-worker-w0-9a5623-6fbf554f99-kwmhs\" (UID: \"16f9970d-5785-4708-a03a-34fc692d3d2a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-9a5623-6fbf554f99-kwmhs"
Oct 30 21:50:35.934951 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:35.934854    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qcjr\" (UniqueName: \"kubernetes.io/projected/80d68d4f-bef1-438c-a194-45506f62291c-kube-api-access-4qcjr\") pod \"payload-processing-e2e-worker-w0-9a5623-597c949cc5-xjcgz\" (UID: \"80d68d4f-bef1-438c-a194-45506f62291c\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-9a5623-597c949cc5-xjcgz"
Oct 30 21:50:35.934951 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:35.934907    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80d68d4f-bef1-438c-a194-45506f62291c-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-9a5623-597c949cc5-xjcgz\" (UID: \"80d68d4f-bef1-438c-a194-45506f62291c\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-9a5623-597c949cc5-xjcgz"
Oct 30 21:50:35.935892 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:35.935868    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80d68d4f-bef1-438c-a194-45506f62291c-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-9a5623-597c949cc5-xjcgz\" (UID: \"80d68d4f-bef1-438c-a194-45506f62291c\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-9a5623-597c949cc5-xjcgz"
Oct 30 21:50:35.975925 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:35.975889    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-9a5623-6fbf554f99-kwmhs"
Oct 30 21:50:35.993226 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:35.993197    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qcjr\" (UniqueName: \"kubernetes.io/projected/80d68d4f-bef1-438c-a194-45506f62291c-kube-api-access-4qcjr\") pod \"payload-processing-e2e-worker-w0-9a5623-597c949cc5-xjcgz\" (UID: \"80d68d4f-bef1-438c-a194-45506f62291c\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-9a5623-597c949cc5-xjcgz"
Oct 30 21:50:36.126300 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:36.126264    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-9a5623-597c949cc5-xjcgz"
Oct 30 21:50:36.220540 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:36.220456    4871 generic.go:356] "Generic (PLEG): container finished" podID="e626a30e-ff4e-4e87-9f50-000865cfe6c0" containerID="94fa926fb003c56e36a68c5a8168d7eafa9f6658ce73dcbfd25763738d50376b" exitCode=0
Oct 30 21:50:36.220675 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:36.220541    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn" event={"ID":"e626a30e-ff4e-4e87-9f50-000865cfe6c0","Type":"ContainerDied","Data":"94fa926fb003c56e36a68c5a8168d7eafa9f6658ce73dcbfd25763738d50376b"}
Oct 30 21:50:36.220950 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:36.220935    4871 scope.go:122] "RemoveContainer" containerID="94fa926fb003c56e36a68c5a8168d7eafa9f6658ce73dcbfd25763738d50376b"
Oct 30 21:50:36.222248 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:36.222211    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:50:36.222341 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:36.222290    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:50:36.222341 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:36.222327    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:50:36.278067 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:36.278029    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-ffe7a1cf-6976c8c597-t5ncj" podUID="a11db216-47ce-442a-adb9-ca7e682eb87e" containerName="payload-processing" containerID="cri-o://113a90e81e3ab67714ef34fc605d174c66190378e3bc931c99ef45f7b66f09eb" gracePeriod=25
Oct 30 21:50:36.379723 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:36.379679    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-9a5623-6fbf554f99-kwmhs"]
Oct 30 21:50:36.386859 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:50:36.386823    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16f9970d_5785_4708_a03a_34fc692d3d2a.slice/crio-3da8f74e77d7f6e6981e7e98861e294303944a3554052789a19cd1da0c691df0 WatchSource:0}: Error finding container 3da8f74e77d7f6e6981e7e98861e294303944a3554052789a19cd1da0c691df0: Status 404 returned error can't find the container with id 3da8f74e77d7f6e6981e7e98861e294303944a3554052789a19cd1da0c691df0
Oct 30 21:50:36.502184 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:36.502134    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-9a5623-597c949cc5-xjcgz"]
Oct 30 21:50:36.572473 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:36.572444    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-ffe7a1cf-6976c8c597-t5ncj"
Oct 30 21:50:36.742141 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:36.742038    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a11db216-47ce-442a-adb9-ca7e682eb87e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a11db216-47ce-442a-adb9-ca7e682eb87e-plugins-config-volume\") pod \"a11db216-47ce-442a-adb9-ca7e682eb87e\" (UID: \"a11db216-47ce-442a-adb9-ca7e682eb87e\") "
Oct 30 21:50:36.742141 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:36.742129    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a11db216-47ce-442a-adb9-ca7e682eb87e-kube-api-access-f4jnt\" (UniqueName: \"kubernetes.io/projected/a11db216-47ce-442a-adb9-ca7e682eb87e-kube-api-access-f4jnt\") pod \"a11db216-47ce-442a-adb9-ca7e682eb87e\" (UID: \"a11db216-47ce-442a-adb9-ca7e682eb87e\") "
Oct 30 21:50:36.742564 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:36.742533    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a11db216-47ce-442a-adb9-ca7e682eb87e-plugins-config-volume" pod "a11db216-47ce-442a-adb9-ca7e682eb87e" (UID: "a11db216-47ce-442a-adb9-ca7e682eb87e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:50:36.744492 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:36.744465    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a11db216-47ce-442a-adb9-ca7e682eb87e-kube-api-access-f4jnt" pod "a11db216-47ce-442a-adb9-ca7e682eb87e" (UID: "a11db216-47ce-442a-adb9-ca7e682eb87e"). InnerVolumeSpecName "kube-api-access-f4jnt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:50:36.843258 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:36.843187    4871 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a11db216-47ce-442a-adb9-ca7e682eb87e-plugins-config-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:36.843258 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:36.843253    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4jnt\" (UniqueName: \"kubernetes.io/projected/a11db216-47ce-442a-adb9-ca7e682eb87e-kube-api-access-f4jnt\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:37.234986 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:37.234943    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn" event={"ID":"e626a30e-ff4e-4e87-9f50-000865cfe6c0","Type":"ContainerStarted","Data":"5ce4de346035374e8b7dc73b17db5e80190c1d1c89b9b8e479c2bf6da38cb474"}
Oct 30 21:50:37.236607 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:37.236561    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-9a5623-597c949cc5-xjcgz" event={"ID":"80d68d4f-bef1-438c-a194-45506f62291c","Type":"ContainerStarted","Data":"e3f28b194ce20f54d391eaf76e2c2d15bed5cb344c5021c9cde864f28062c9d2"}
Oct 30 21:50:37.236738 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:37.236628    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-9a5623-597c949cc5-xjcgz" event={"ID":"80d68d4f-bef1-438c-a194-45506f62291c","Type":"ContainerStarted","Data":"d9ea43ed7ea81d0d9abbf41d977a1d8f5ab3516bc4864e9a6cc745d82dbe861d"}
Oct 30 21:50:37.236738 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:37.236674    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-9a5623-597c949cc5-xjcgz"
Oct 30 21:50:37.237949 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:37.237928    4871 generic.go:356] "Generic (PLEG): container finished" podID="a11db216-47ce-442a-adb9-ca7e682eb87e" containerID="113a90e81e3ab67714ef34fc605d174c66190378e3bc931c99ef45f7b66f09eb" exitCode=0
Oct 30 21:50:37.238064 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:37.237960    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-ffe7a1cf-6976c8c597-t5ncj"
Oct 30 21:50:37.238064 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:37.238002    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-ffe7a1cf-6976c8c597-t5ncj" event={"ID":"a11db216-47ce-442a-adb9-ca7e682eb87e","Type":"ContainerDied","Data":"113a90e81e3ab67714ef34fc605d174c66190378e3bc931c99ef45f7b66f09eb"}
Oct 30 21:50:37.238064 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:37.238029    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-ffe7a1cf-6976c8c597-t5ncj" event={"ID":"a11db216-47ce-442a-adb9-ca7e682eb87e","Type":"ContainerDied","Data":"909b029ab4106eecb80534d50bdc9c58d42b789fec47f312b21135ef410cb28c"}
Oct 30 21:50:37.238064 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:37.238050    4871 scope.go:122] "RemoveContainer" containerID="113a90e81e3ab67714ef34fc605d174c66190378e3bc931c99ef45f7b66f09eb"
Oct 30 21:50:37.239876 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:37.239850    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-9a5623-6fbf554f99-kwmhs" event={"ID":"16f9970d-5785-4708-a03a-34fc692d3d2a","Type":"ContainerStarted","Data":"9774fbd0fcd6d672c70abbc260aebd14609eed0846424550948e7252cd180084"}
Oct 30 21:50:37.239971 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:37.239899    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-9a5623-6fbf554f99-kwmhs" event={"ID":"16f9970d-5785-4708-a03a-34fc692d3d2a","Type":"ContainerStarted","Data":"3da8f74e77d7f6e6981e7e98861e294303944a3554052789a19cd1da0c691df0"}
Oct 30 21:50:37.240015 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:37.239991    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-9a5623-6fbf554f99-kwmhs"
Oct 30 21:50:37.249983 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:37.249966    4871 scope.go:122] "RemoveContainer" containerID="113a90e81e3ab67714ef34fc605d174c66190378e3bc931c99ef45f7b66f09eb"
Oct 30 21:50:37.250281 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:50:37.250223    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"113a90e81e3ab67714ef34fc605d174c66190378e3bc931c99ef45f7b66f09eb\": container with ID starting with 113a90e81e3ab67714ef34fc605d174c66190378e3bc931c99ef45f7b66f09eb not found: ID does not exist" containerID="113a90e81e3ab67714ef34fc605d174c66190378e3bc931c99ef45f7b66f09eb"
Oct 30 21:50:37.250342 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:37.250268    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"113a90e81e3ab67714ef34fc605d174c66190378e3bc931c99ef45f7b66f09eb"} err="failed to get container status \"113a90e81e3ab67714ef34fc605d174c66190378e3bc931c99ef45f7b66f09eb\": rpc error: code = NotFound desc = could not find container \"113a90e81e3ab67714ef34fc605d174c66190378e3bc931c99ef45f7b66f09eb\": container with ID starting with 113a90e81e3ab67714ef34fc605d174c66190378e3bc931c99ef45f7b66f09eb not found: ID does not exist"
Oct 30 21:50:37.455776 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:37.455734    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:37.461054 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:37.461001    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-9a5623-6fbf554f99-kwmhs" podStartSLOduration=2.460989249 podStartE2EDuration="2.460989249s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:50:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:50:37.460357369 +0000 UTC m=+779.637845985" watchObservedRunningTime="2026-10-30 21:50:37.460989249 +0000 UTC m=+779.638477866"
Oct 30 21:50:37.481865 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:37.481831    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"
Oct 30 21:50:37.482303 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:37.482279    4871 patch_prober.go:29] interesting pod/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 30 21:50:37.482363 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:37.482329    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn" podUID="e626a30e-ff4e-4e87-9f50-000865cfe6c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 30 21:50:37.573141 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:37.573048    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-ffe7a1cf-6976c8c597-t5ncj"]
Oct 30 21:50:37.580267 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:37.576998    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-ffe7a1cf-6976c8c597-t5ncj"]
Oct 30 21:50:37.958964 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:37.958933    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a11db216-47ce-442a-adb9-ca7e682eb87e" path="/var/lib/kubelet/pods/a11db216-47ce-442a-adb9-ca7e682eb87e/volumes"
Oct 30 21:50:38.482034 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:38.482002    4871 patch_prober.go:29] interesting pod/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 30 21:50:38.482428 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:38.482067    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn" podUID="e626a30e-ff4e-4e87-9f50-000865cfe6c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 30 21:50:38.550750 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:38.550721    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:38.742321 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:38.720648    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:38.860686 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:38.860651    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"
Oct 30 21:50:38.861181 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:38.861157    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d11e9475-1d27-44b3-b433-13a5cc49a13a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d11e9475-1d27-44b3-b433-13a5cc49a13a-workload-socket\") pod \"d11e9475-1d27-44b3-b433-13a5cc49a13a\" (UID: \"d11e9475-1d27-44b3-b433-13a5cc49a13a\") "
Oct 30 21:50:38.861313 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:38.861196    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d11e9475-1d27-44b3-b433-13a5cc49a13a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d11e9475-1d27-44b3-b433-13a5cc49a13a-istio-envoy\") pod \"d11e9475-1d27-44b3-b433-13a5cc49a13a\" (UID: \"d11e9475-1d27-44b3-b433-13a5cc49a13a\") "
Oct 30 21:50:38.861313 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:38.861223    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d11e9475-1d27-44b3-b433-13a5cc49a13a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d11e9475-1d27-44b3-b433-13a5cc49a13a-istiod-ca-cert\") pod \"d11e9475-1d27-44b3-b433-13a5cc49a13a\" (UID: \"d11e9475-1d27-44b3-b433-13a5cc49a13a\") "
Oct 30 21:50:38.861313 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:38.861285    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d11e9475-1d27-44b3-b433-13a5cc49a13a-kube-api-access-z8gjm\" (UniqueName: \"kubernetes.io/projected/d11e9475-1d27-44b3-b433-13a5cc49a13a-kube-api-access-z8gjm\") pod \"d11e9475-1d27-44b3-b433-13a5cc49a13a\" (UID: \"d11e9475-1d27-44b3-b433-13a5cc49a13a\") "
Oct 30 21:50:38.861459 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:38.861318    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d11e9475-1d27-44b3-b433-13a5cc49a13a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d11e9475-1d27-44b3-b433-13a5cc49a13a-istio-data\") pod \"d11e9475-1d27-44b3-b433-13a5cc49a13a\" (UID: \"d11e9475-1d27-44b3-b433-13a5cc49a13a\") "
Oct 30 21:50:38.861459 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:38.861346    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d11e9475-1d27-44b3-b433-13a5cc49a13a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d11e9475-1d27-44b3-b433-13a5cc49a13a-credential-socket\") pod \"d11e9475-1d27-44b3-b433-13a5cc49a13a\" (UID: \"d11e9475-1d27-44b3-b433-13a5cc49a13a\") "
Oct 30 21:50:38.861459 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:38.861388    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d11e9475-1d27-44b3-b433-13a5cc49a13a-istio-token\" (UniqueName: \"kubernetes.io/projected/d11e9475-1d27-44b3-b433-13a5cc49a13a-istio-token\") pod \"d11e9475-1d27-44b3-b433-13a5cc49a13a\" (UID: \"d11e9475-1d27-44b3-b433-13a5cc49a13a\") "
Oct 30 21:50:38.861459 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:38.861421    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d11e9475-1d27-44b3-b433-13a5cc49a13a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d11e9475-1d27-44b3-b433-13a5cc49a13a-workload-certs\") pod \"d11e9475-1d27-44b3-b433-13a5cc49a13a\" (UID: \"d11e9475-1d27-44b3-b433-13a5cc49a13a\") "
Oct 30 21:50:38.861710 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:38.861463    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d11e9475-1d27-44b3-b433-13a5cc49a13a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d11e9475-1d27-44b3-b433-13a5cc49a13a-istio-podinfo\") pod \"d11e9475-1d27-44b3-b433-13a5cc49a13a\" (UID: \"d11e9475-1d27-44b3-b433-13a5cc49a13a\") "
Oct 30 21:50:38.862279 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:38.862226    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d11e9475-1d27-44b3-b433-13a5cc49a13a-workload-socket" pod "d11e9475-1d27-44b3-b433-13a5cc49a13a" (UID: "d11e9475-1d27-44b3-b433-13a5cc49a13a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:38.862401 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:38.862314    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d11e9475-1d27-44b3-b433-13a5cc49a13a-credential-socket" pod "d11e9475-1d27-44b3-b433-13a5cc49a13a" (UID: "d11e9475-1d27-44b3-b433-13a5cc49a13a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:38.862527 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:38.862508    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d11e9475-1d27-44b3-b433-13a5cc49a13a-istio-data" pod "d11e9475-1d27-44b3-b433-13a5cc49a13a" (UID: "d11e9475-1d27-44b3-b433-13a5cc49a13a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:38.862585 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:38.862536    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d11e9475-1d27-44b3-b433-13a5cc49a13a-istiod-ca-cert" pod "d11e9475-1d27-44b3-b433-13a5cc49a13a" (UID: "d11e9475-1d27-44b3-b433-13a5cc49a13a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:50:38.862804 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:38.862781    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d11e9475-1d27-44b3-b433-13a5cc49a13a-workload-certs" pod "d11e9475-1d27-44b3-b433-13a5cc49a13a" (UID: "d11e9475-1d27-44b3-b433-13a5cc49a13a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:38.866490 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:38.866431    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d11e9475-1d27-44b3-b433-13a5cc49a13a-kube-api-access-z8gjm" pod "d11e9475-1d27-44b3-b433-13a5cc49a13a" (UID: "d11e9475-1d27-44b3-b433-13a5cc49a13a"). InnerVolumeSpecName "kube-api-access-z8gjm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:50:38.866490 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:38.866450    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d11e9475-1d27-44b3-b433-13a5cc49a13a-istio-token" pod "d11e9475-1d27-44b3-b433-13a5cc49a13a" (UID: "d11e9475-1d27-44b3-b433-13a5cc49a13a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:50:38.866490 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:38.866471    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d11e9475-1d27-44b3-b433-13a5cc49a13a-istio-envoy" pod "d11e9475-1d27-44b3-b433-13a5cc49a13a" (UID: "d11e9475-1d27-44b3-b433-13a5cc49a13a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:38.867061 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:38.866785    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d11e9475-1d27-44b3-b433-13a5cc49a13a-istio-podinfo" pod "d11e9475-1d27-44b3-b433-13a5cc49a13a" (UID: "d11e9475-1d27-44b3-b433-13a5cc49a13a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:50:38.891597 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:38.891572    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:38.962765 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:38.962732    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d11e9475-1d27-44b3-b433-13a5cc49a13a-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:38.962765 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:38.962761    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d11e9475-1d27-44b3-b433-13a5cc49a13a-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:38.962765 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:38.962770    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d11e9475-1d27-44b3-b433-13a5cc49a13a-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:38.962765 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:38.962780    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d11e9475-1d27-44b3-b433-13a5cc49a13a-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:38.962765 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:38.962789    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8gjm\" (UniqueName: \"kubernetes.io/projected/d11e9475-1d27-44b3-b433-13a5cc49a13a-kube-api-access-z8gjm\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:38.963128 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:38.962798    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d11e9475-1d27-44b3-b433-13a5cc49a13a-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:38.963128 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:38.962810    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d11e9475-1d27-44b3-b433-13a5cc49a13a-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:38.963128 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:38.962822    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d11e9475-1d27-44b3-b433-13a5cc49a13a-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:38.963128 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:38.962830    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d11e9475-1d27-44b3-b433-13a5cc49a13a-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:39.038318 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:39.037705    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-9a5623-597c949cc5-xjcgz" podStartSLOduration=4.037681859 podStartE2EDuration="4.037681859s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:50:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:50:37.69541432 +0000 UTC m=+779.872902936" watchObservedRunningTime="2026-10-30 21:50:39.037681859 +0000 UTC m=+781.215170478"
Oct 30 21:50:39.255346 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:39.255311    4871 generic.go:356] "Generic (PLEG): container finished" podID="d11e9475-1d27-44b3-b433-13a5cc49a13a" containerID="bee8fa33b605bcadb1c43011eb60ffe6a41f4c9d75be42d381c5721e9f02c800" exitCode=0
Oct 30 21:50:39.255511 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:39.255369    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"
Oct 30 21:50:39.255511 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:39.255384    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw" event={"ID":"d11e9475-1d27-44b3-b433-13a5cc49a13a","Type":"ContainerDied","Data":"bee8fa33b605bcadb1c43011eb60ffe6a41f4c9d75be42d381c5721e9f02c800"}
Oct 30 21:50:39.255511 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:39.255427    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw" event={"ID":"d11e9475-1d27-44b3-b433-13a5cc49a13a","Type":"ContainerDied","Data":"e32a7b1eee2788a161e819c40baa44044b9bcfb28da18e62e41f6d73accfd953"}
Oct 30 21:50:39.255511 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:39.255448    4871 scope.go:122] "RemoveContainer" containerID="bee8fa33b605bcadb1c43011eb60ffe6a41f4c9d75be42d381c5721e9f02c800"
Oct 30 21:50:39.271226 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:39.271207    4871 scope.go:122] "RemoveContainer" containerID="948826c218be85302c71f09ea7e45d8636ac493c23c162e3f1d7c4a1cfcf6c75"
Oct 30 21:50:39.287130 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:39.287095    4871 scope.go:122] "RemoveContainer" containerID="bee8fa33b605bcadb1c43011eb60ffe6a41f4c9d75be42d381c5721e9f02c800"
Oct 30 21:50:39.287715 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:50:39.287681    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bee8fa33b605bcadb1c43011eb60ffe6a41f4c9d75be42d381c5721e9f02c800\": container with ID starting with bee8fa33b605bcadb1c43011eb60ffe6a41f4c9d75be42d381c5721e9f02c800 not found: ID does not exist" containerID="bee8fa33b605bcadb1c43011eb60ffe6a41f4c9d75be42d381c5721e9f02c800"
Oct 30 21:50:39.287925 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:39.287850    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bee8fa33b605bcadb1c43011eb60ffe6a41f4c9d75be42d381c5721e9f02c800"} err="failed to get container status \"bee8fa33b605bcadb1c43011eb60ffe6a41f4c9d75be42d381c5721e9f02c800\": rpc error: code = NotFound desc = could not find container \"bee8fa33b605bcadb1c43011eb60ffe6a41f4c9d75be42d381c5721e9f02c800\": container with ID starting with bee8fa33b605bcadb1c43011eb60ffe6a41f4c9d75be42d381c5721e9f02c800 not found: ID does not exist"
Oct 30 21:50:39.287925 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:39.287908    4871 scope.go:122] "RemoveContainer" containerID="948826c218be85302c71f09ea7e45d8636ac493c23c162e3f1d7c4a1cfcf6c75"
Oct 30 21:50:39.288618 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:50:39.288562    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"948826c218be85302c71f09ea7e45d8636ac493c23c162e3f1d7c4a1cfcf6c75\": container with ID starting with 948826c218be85302c71f09ea7e45d8636ac493c23c162e3f1d7c4a1cfcf6c75 not found: ID does not exist" containerID="948826c218be85302c71f09ea7e45d8636ac493c23c162e3f1d7c4a1cfcf6c75"
Oct 30 21:50:39.288707 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:39.288599    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"948826c218be85302c71f09ea7e45d8636ac493c23c162e3f1d7c4a1cfcf6c75"} err="failed to get container status \"948826c218be85302c71f09ea7e45d8636ac493c23c162e3f1d7c4a1cfcf6c75\": rpc error: code = NotFound desc = could not find container \"948826c218be85302c71f09ea7e45d8636ac493c23c162e3f1d7c4a1cfcf6c75\": container with ID starting with 948826c218be85302c71f09ea7e45d8636ac493c23c162e3f1d7c4a1cfcf6c75 not found: ID does not exist"
Oct 30 21:50:39.301290 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:39.301264    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:39.468317 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:39.462295    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"]
Oct 30 21:50:39.468317 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:39.468022    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-a1d107b0-openshift-default-5646bbf8d5-xbqsw"]
Oct 30 21:50:39.482093 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:39.482058    4871 patch_prober.go:29] interesting pod/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 30 21:50:39.482609 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:39.482105    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn" podUID="e626a30e-ff4e-4e87-9f50-000865cfe6c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 30 21:50:39.593177 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:39.593079    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:39.796759 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:39.796714    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-16c49820-z8bwc"]
Oct 30 21:50:39.799597 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:39.799569    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-16c49820-z8bwc"]
Oct 30 21:50:39.887648 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:39.887615    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"]
Oct 30 21:50:39.888282 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:39.887945    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn" podUID="e626a30e-ff4e-4e87-9f50-000865cfe6c0" containerName="istio-proxy" containerID="cri-o://5ce4de346035374e8b7dc73b17db5e80190c1d1c89b9b8e479c2bf6da38cb474" gracePeriod=30
Oct 30 21:50:39.935522 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:39.935402    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:39.960517 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:39.960484    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cdcbb401-ed43-47af-9f55-0123c8b19e6e" path="/var/lib/kubelet/pods/cdcbb401-ed43-47af-9f55-0123c8b19e6e/volumes"
Oct 30 21:50:39.960930 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:39.960916    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d11e9475-1d27-44b3-b433-13a5cc49a13a" path="/var/lib/kubelet/pods/d11e9475-1d27-44b3-b433-13a5cc49a13a/volumes"
Oct 30 21:50:40.455551 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:40.453692    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:40.652350 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:40.652302    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:40.811315 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:40.811167    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:41.134060 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:41.133987    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:41.390552 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:41.390468    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:41.572302 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:41.572260    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ffe7a1cf-kl9xs"]
Oct 30 21:50:41.647290 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:41.647184    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ffe7a1cf-kl9xs"]
Oct 30 21:50:41.682424 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:41.682387    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"]
Oct 30 21:50:41.682935 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:41.682741    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg" podUID="03d16436-58d3-495e-84d4-38b6d3b55a8b" containerName="istio-proxy" containerID="cri-o://6cf1c83712e25baed3812e82ef12b839c26496846ddb9ae779273fdd266a9f30" gracePeriod=30
Oct 30 21:50:41.723263 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:41.702483    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:41.960297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:41.960210    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="df948643-feaf-4ed8-b5e9-9e2e5e075e83" path="/var/lib/kubelet/pods/df948643-feaf-4ed8-b5e9-9e2e5e075e83/volumes"
Oct 30 21:50:41.975630 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:41.975597    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:42.177283 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:42.177248    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:42.371429 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:42.371388    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:44.181361 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:44.181313    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:44.367146 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:44.367099    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:44.614219 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:44.614183    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-26743a-75976bb55f-5vl8h"]
Oct 30 21:50:44.614650 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:44.614636    4871 cpu_manager.go:414] "containerMap: removing container" podUID="d11e9475-1d27-44b3-b433-13a5cc49a13a" containerName="istio-proxy"
Oct 30 21:50:44.614693 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:44.614651    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d11e9475-1d27-44b3-b433-13a5cc49a13a" containerName="istio-proxy"
Oct 30 21:50:44.614693 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:44.614666    4871 cpu_manager.go:414] "containerMap: removing container" podUID="a11db216-47ce-442a-adb9-ca7e682eb87e" containerName="payload-processing"
Oct 30 21:50:44.614693 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:44.614672    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a11db216-47ce-442a-adb9-ca7e682eb87e" containerName="payload-processing"
Oct 30 21:50:44.614693 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:44.614678    4871 cpu_manager.go:414] "containerMap: removing container" podUID="d11e9475-1d27-44b3-b433-13a5cc49a13a" containerName="istio-proxy"
Oct 30 21:50:44.614693 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:44.614683    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d11e9475-1d27-44b3-b433-13a5cc49a13a" containerName="istio-proxy"
Oct 30 21:50:44.614844 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:44.614763    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="d11e9475-1d27-44b3-b433-13a5cc49a13a" containerName="istio-proxy"
Oct 30 21:50:44.614844 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:44.614772    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="a11db216-47ce-442a-adb9-ca7e682eb87e" containerName="payload-processing"
Oct 30 21:50:44.617993 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:44.617973    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-26743a-75976bb55f-5vl8h"
Oct 30 21:50:44.673206 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:44.673164    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-26743a-75976bb55f-5vl8h"]
Oct 30 21:50:44.725558 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:44.725506    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-26743a\""
Oct 30 21:50:44.725821 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:44.725790    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcqtm\" (UniqueName: \"kubernetes.io/projected/9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c-kube-api-access-vcqtm\") pod \"payload-pre-processing-e2e-worker-w4-26743a-75976bb55f-5vl8h\" (UID: \"9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-26743a-75976bb55f-5vl8h"
Oct 30 21:50:44.725963 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:44.725944    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-26743a-75976bb55f-5vl8h\" (UID: \"9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-26743a-75976bb55f-5vl8h"
Oct 30 21:50:44.726265 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:44.726204    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-26743a-dockercfg-zsb9g\""
Oct 30 21:50:44.733758 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:44.733720    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-26743a-fcb59c9c7-s4rnl"]
Oct 30 21:50:44.734367 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:44.734348    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="d11e9475-1d27-44b3-b433-13a5cc49a13a" containerName="istio-proxy"
Oct 30 21:50:44.736552 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:44.736529    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-26743a-fcb59c9c7-s4rnl"
Oct 30 21:50:44.797485 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:44.797447    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-26743a-fcb59c9c7-s4rnl"]
Oct 30 21:50:44.826975 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:44.826935    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff4cdc4a-dff3-4e33-beae-ce10bb079a80-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-26743a-fcb59c9c7-s4rnl\" (UID: \"ff4cdc4a-dff3-4e33-beae-ce10bb079a80\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-26743a-fcb59c9c7-s4rnl"
Oct 30 21:50:44.827175 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:44.827075    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-26743a-75976bb55f-5vl8h\" (UID: \"9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-26743a-75976bb55f-5vl8h"
Oct 30 21:50:44.827271 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:44.827197    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5hbc\" (UniqueName: \"kubernetes.io/projected/ff4cdc4a-dff3-4e33-beae-ce10bb079a80-kube-api-access-l5hbc\") pod \"payload-processing-e2e-worker-w4-26743a-fcb59c9c7-s4rnl\" (UID: \"ff4cdc4a-dff3-4e33-beae-ce10bb079a80\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-26743a-fcb59c9c7-s4rnl"
Oct 30 21:50:44.827345 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:44.827287    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcqtm\" (UniqueName: \"kubernetes.io/projected/9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c-kube-api-access-vcqtm\") pod \"payload-pre-processing-e2e-worker-w4-26743a-75976bb55f-5vl8h\" (UID: \"9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-26743a-75976bb55f-5vl8h"
Oct 30 21:50:44.828003 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:44.827977    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-26743a-75976bb55f-5vl8h\" (UID: \"9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-26743a-75976bb55f-5vl8h"
Oct 30 21:50:44.885540 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:44.885460    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcqtm\" (UniqueName: \"kubernetes.io/projected/9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c-kube-api-access-vcqtm\") pod \"payload-pre-processing-e2e-worker-w4-26743a-75976bb55f-5vl8h\" (UID: \"9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-26743a-75976bb55f-5vl8h"
Oct 30 21:50:44.928567 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:44.928530    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5hbc\" (UniqueName: \"kubernetes.io/projected/ff4cdc4a-dff3-4e33-beae-ce10bb079a80-kube-api-access-l5hbc\") pod \"payload-processing-e2e-worker-w4-26743a-fcb59c9c7-s4rnl\" (UID: \"ff4cdc4a-dff3-4e33-beae-ce10bb079a80\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-26743a-fcb59c9c7-s4rnl"
Oct 30 21:50:44.928728 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:44.928594    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff4cdc4a-dff3-4e33-beae-ce10bb079a80-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-26743a-fcb59c9c7-s4rnl\" (UID: \"ff4cdc4a-dff3-4e33-beae-ce10bb079a80\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-26743a-fcb59c9c7-s4rnl"
Oct 30 21:50:44.929376 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:44.929357    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff4cdc4a-dff3-4e33-beae-ce10bb079a80-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-26743a-fcb59c9c7-s4rnl\" (UID: \"ff4cdc4a-dff3-4e33-beae-ce10bb079a80\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-26743a-fcb59c9c7-s4rnl"
Oct 30 21:50:44.930369 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:44.930345    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-26743a-75976bb55f-5vl8h"
Oct 30 21:50:44.987494 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:44.987461    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5hbc\" (UniqueName: \"kubernetes.io/projected/ff4cdc4a-dff3-4e33-beae-ce10bb079a80-kube-api-access-l5hbc\") pod \"payload-processing-e2e-worker-w4-26743a-fcb59c9c7-s4rnl\" (UID: \"ff4cdc4a-dff3-4e33-beae-ce10bb079a80\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-26743a-fcb59c9c7-s4rnl"
Oct 30 21:50:45.049679 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:45.049641    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-26743a-fcb59c9c7-s4rnl"
Oct 30 21:50:45.299930 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:45.299901    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-26743a-75976bb55f-5vl8h"]
Oct 30 21:50:45.302899 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:50:45.302870    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dfec38a_aaa5_4884_8fb0_9bfa4efbcb6c.slice/crio-90fbfd425060d773d7f22145d76d6336d63131cf4588a47da09c0097e1db1cca WatchSource:0}: Error finding container 90fbfd425060d773d7f22145d76d6336d63131cf4588a47da09c0097e1db1cca: Status 404 returned error can't find the container with id 90fbfd425060d773d7f22145d76d6336d63131cf4588a47da09c0097e1db1cca
Oct 30 21:50:45.421214 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:45.421186    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-26743a-fcb59c9c7-s4rnl"]
Oct 30 21:50:45.422677 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:50:45.422647    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff4cdc4a_dff3_4e33_beae_ce10bb079a80.slice/crio-83df7e028de913b5109e4b50f36f52c134ab34a09588db832c96ab2f320cc813 WatchSource:0}: Error finding container 83df7e028de913b5109e4b50f36f52c134ab34a09588db832c96ab2f320cc813: Status 404 returned error can't find the container with id 83df7e028de913b5109e4b50f36f52c134ab34a09588db832c96ab2f320cc813
Oct 30 21:50:45.761492 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:45.761455    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:46.311625 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:46.311588    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-26743a-fcb59c9c7-s4rnl" event={"ID":"ff4cdc4a-dff3-4e33-beae-ce10bb079a80","Type":"ContainerStarted","Data":"5e9e0f59cacf0089232321e2abaae89cf0c298a82dcdb0da7aee0c1b0239b8de"}
Oct 30 21:50:46.312111 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:46.311765    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-26743a-fcb59c9c7-s4rnl"
Oct 30 21:50:46.312111 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:46.311810    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-26743a-fcb59c9c7-s4rnl" event={"ID":"ff4cdc4a-dff3-4e33-beae-ce10bb079a80","Type":"ContainerStarted","Data":"83df7e028de913b5109e4b50f36f52c134ab34a09588db832c96ab2f320cc813"}
Oct 30 21:50:46.313266 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:46.313227    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-26743a-75976bb55f-5vl8h" event={"ID":"9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c","Type":"ContainerStarted","Data":"ebceb8b74027404ef3569a82d81e803020729f8925c67ec748db13d39979a7ce"}
Oct 30 21:50:46.313429 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:46.313280    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-26743a-75976bb55f-5vl8h" event={"ID":"9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c","Type":"ContainerStarted","Data":"90fbfd425060d773d7f22145d76d6336d63131cf4588a47da09c0097e1db1cca"}
Oct 30 21:50:46.313429 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:46.313306    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-26743a-75976bb55f-5vl8h"
Oct 30 21:50:46.423937 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:46.423871    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-26743a-fcb59c9c7-s4rnl" podStartSLOduration=2.423851438 podStartE2EDuration="2.423851438s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:50:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:50:46.42054528 +0000 UTC m=+788.598033897" watchObservedRunningTime="2026-10-30 21:50:46.423851438 +0000 UTC m=+788.601340056"
Oct 30 21:50:46.531641 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:46.531584    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-26743a-75976bb55f-5vl8h" podStartSLOduration=2.531569055 podStartE2EDuration="2.531569055s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:50:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:50:46.529579832 +0000 UTC m=+788.707068449" watchObservedRunningTime="2026-10-30 21:50:46.531569055 +0000 UTC m=+788.709057672"
Oct 30 21:50:46.947395 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:46.947329    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"
Oct 30 21:50:47.053159 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.053115    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/03d16436-58d3-495e-84d4-38b6d3b55a8b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03d16436-58d3-495e-84d4-38b6d3b55a8b-istiod-ca-cert\") pod \"03d16436-58d3-495e-84d4-38b6d3b55a8b\" (UID: \"03d16436-58d3-495e-84d4-38b6d3b55a8b\") "
Oct 30 21:50:47.053378 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.053191    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03d16436-58d3-495e-84d4-38b6d3b55a8b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03d16436-58d3-495e-84d4-38b6d3b55a8b-credential-socket\") pod \"03d16436-58d3-495e-84d4-38b6d3b55a8b\" (UID: \"03d16436-58d3-495e-84d4-38b6d3b55a8b\") "
Oct 30 21:50:47.053378 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.053225    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/03d16436-58d3-495e-84d4-38b6d3b55a8b-kube-api-access-v2vdh\" (UniqueName: \"kubernetes.io/projected/03d16436-58d3-495e-84d4-38b6d3b55a8b-kube-api-access-v2vdh\") pod \"03d16436-58d3-495e-84d4-38b6d3b55a8b\" (UID: \"03d16436-58d3-495e-84d4-38b6d3b55a8b\") "
Oct 30 21:50:47.053378 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.053285    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03d16436-58d3-495e-84d4-38b6d3b55a8b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03d16436-58d3-495e-84d4-38b6d3b55a8b-istio-data\") pod \"03d16436-58d3-495e-84d4-38b6d3b55a8b\" (UID: \"03d16436-58d3-495e-84d4-38b6d3b55a8b\") "
Oct 30 21:50:47.053378 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.053332    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/03d16436-58d3-495e-84d4-38b6d3b55a8b-istio-token\" (UniqueName: \"kubernetes.io/projected/03d16436-58d3-495e-84d4-38b6d3b55a8b-istio-token\") pod \"03d16436-58d3-495e-84d4-38b6d3b55a8b\" (UID: \"03d16436-58d3-495e-84d4-38b6d3b55a8b\") "
Oct 30 21:50:47.053601 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.053410    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03d16436-58d3-495e-84d4-38b6d3b55a8b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03d16436-58d3-495e-84d4-38b6d3b55a8b-workload-socket\") pod \"03d16436-58d3-495e-84d4-38b6d3b55a8b\" (UID: \"03d16436-58d3-495e-84d4-38b6d3b55a8b\") "
Oct 30 21:50:47.053601 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.053442    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03d16436-58d3-495e-84d4-38b6d3b55a8b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03d16436-58d3-495e-84d4-38b6d3b55a8b-istio-envoy\") pod \"03d16436-58d3-495e-84d4-38b6d3b55a8b\" (UID: \"03d16436-58d3-495e-84d4-38b6d3b55a8b\") "
Oct 30 21:50:47.053601 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.053517    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03d16436-58d3-495e-84d4-38b6d3b55a8b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03d16436-58d3-495e-84d4-38b6d3b55a8b-workload-certs\") pod \"03d16436-58d3-495e-84d4-38b6d3b55a8b\" (UID: \"03d16436-58d3-495e-84d4-38b6d3b55a8b\") "
Oct 30 21:50:47.053601 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.053546    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03d16436-58d3-495e-84d4-38b6d3b55a8b-credential-socket" pod "03d16436-58d3-495e-84d4-38b6d3b55a8b" (UID: "03d16436-58d3-495e-84d4-38b6d3b55a8b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:47.053601 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.053553    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/03d16436-58d3-495e-84d4-38b6d3b55a8b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03d16436-58d3-495e-84d4-38b6d3b55a8b-istio-podinfo\") pod \"03d16436-58d3-495e-84d4-38b6d3b55a8b\" (UID: \"03d16436-58d3-495e-84d4-38b6d3b55a8b\") "
Oct 30 21:50:47.053843 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.053673    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03d16436-58d3-495e-84d4-38b6d3b55a8b-istio-data" pod "03d16436-58d3-495e-84d4-38b6d3b55a8b" (UID: "03d16436-58d3-495e-84d4-38b6d3b55a8b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:47.053843 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.053587    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03d16436-58d3-495e-84d4-38b6d3b55a8b-istiod-ca-cert" pod "03d16436-58d3-495e-84d4-38b6d3b55a8b" (UID: "03d16436-58d3-495e-84d4-38b6d3b55a8b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:50:47.053946 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.053881    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03d16436-58d3-495e-84d4-38b6d3b55a8b-workload-socket" pod "03d16436-58d3-495e-84d4-38b6d3b55a8b" (UID: "03d16436-58d3-495e-84d4-38b6d3b55a8b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:47.053996 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.053980    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03d16436-58d3-495e-84d4-38b6d3b55a8b-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:47.054051 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.053999    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03d16436-58d3-495e-84d4-38b6d3b55a8b-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:47.054051 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.054015    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03d16436-58d3-495e-84d4-38b6d3b55a8b-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:47.054051 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.054031    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03d16436-58d3-495e-84d4-38b6d3b55a8b-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:47.054051 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.053988    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03d16436-58d3-495e-84d4-38b6d3b55a8b-workload-certs" pod "03d16436-58d3-495e-84d4-38b6d3b55a8b" (UID: "03d16436-58d3-495e-84d4-38b6d3b55a8b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:47.056140 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.056110    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/03d16436-58d3-495e-84d4-38b6d3b55a8b-istio-podinfo" pod "03d16436-58d3-495e-84d4-38b6d3b55a8b" (UID: "03d16436-58d3-495e-84d4-38b6d3b55a8b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:50:47.056310 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.056141    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03d16436-58d3-495e-84d4-38b6d3b55a8b-kube-api-access-v2vdh" pod "03d16436-58d3-495e-84d4-38b6d3b55a8b" (UID: "03d16436-58d3-495e-84d4-38b6d3b55a8b"). InnerVolumeSpecName "kube-api-access-v2vdh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:50:47.056310 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.056193    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03d16436-58d3-495e-84d4-38b6d3b55a8b-istio-envoy" pod "03d16436-58d3-495e-84d4-38b6d3b55a8b" (UID: "03d16436-58d3-495e-84d4-38b6d3b55a8b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:47.056454 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.056407    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03d16436-58d3-495e-84d4-38b6d3b55a8b-istio-token" pod "03d16436-58d3-495e-84d4-38b6d3b55a8b" (UID: "03d16436-58d3-495e-84d4-38b6d3b55a8b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:50:47.155342 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.155305    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2vdh\" (UniqueName: \"kubernetes.io/projected/03d16436-58d3-495e-84d4-38b6d3b55a8b-kube-api-access-v2vdh\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:47.155342 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.155340    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03d16436-58d3-495e-84d4-38b6d3b55a8b-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:47.155342 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.155351    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03d16436-58d3-495e-84d4-38b6d3b55a8b-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:47.155664 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.155360    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03d16436-58d3-495e-84d4-38b6d3b55a8b-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:47.155664 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.155372    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03d16436-58d3-495e-84d4-38b6d3b55a8b-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:47.322413 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.322321    4871 generic.go:356] "Generic (PLEG): container finished" podID="03d16436-58d3-495e-84d4-38b6d3b55a8b" containerID="6cf1c83712e25baed3812e82ef12b839c26496846ddb9ae779273fdd266a9f30" exitCode=0
Oct 30 21:50:47.322413 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.322373    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"
Oct 30 21:50:47.322893 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.322414    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg" event={"ID":"03d16436-58d3-495e-84d4-38b6d3b55a8b","Type":"ContainerDied","Data":"6cf1c83712e25baed3812e82ef12b839c26496846ddb9ae779273fdd266a9f30"}
Oct 30 21:50:47.322893 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.322475    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg" event={"ID":"03d16436-58d3-495e-84d4-38b6d3b55a8b","Type":"ContainerDied","Data":"4cae6cc89ccddea92bad4a3e2feca061190f93b141d2d8cca915986cd82476ae"}
Oct 30 21:50:47.322893 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.322506    4871 scope.go:122] "RemoveContainer" containerID="6cf1c83712e25baed3812e82ef12b839c26496846ddb9ae779273fdd266a9f30"
Oct 30 21:50:47.333159 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.333141    4871 scope.go:122] "RemoveContainer" containerID="ded087dbe4633a08e6dbfe18cf8f74717787fd47a7044fda7a3cb81004168ee5"
Oct 30 21:50:47.342027 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.342002    4871 scope.go:122] "RemoveContainer" containerID="6cf1c83712e25baed3812e82ef12b839c26496846ddb9ae779273fdd266a9f30"
Oct 30 21:50:47.342395 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:50:47.342371    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cf1c83712e25baed3812e82ef12b839c26496846ddb9ae779273fdd266a9f30\": container with ID starting with 6cf1c83712e25baed3812e82ef12b839c26496846ddb9ae779273fdd266a9f30 not found: ID does not exist" containerID="6cf1c83712e25baed3812e82ef12b839c26496846ddb9ae779273fdd266a9f30"
Oct 30 21:50:47.342475 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.342405    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cf1c83712e25baed3812e82ef12b839c26496846ddb9ae779273fdd266a9f30"} err="failed to get container status \"6cf1c83712e25baed3812e82ef12b839c26496846ddb9ae779273fdd266a9f30\": rpc error: code = NotFound desc = could not find container \"6cf1c83712e25baed3812e82ef12b839c26496846ddb9ae779273fdd266a9f30\": container with ID starting with 6cf1c83712e25baed3812e82ef12b839c26496846ddb9ae779273fdd266a9f30 not found: ID does not exist"
Oct 30 21:50:47.342475 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.342424    4871 scope.go:122] "RemoveContainer" containerID="ded087dbe4633a08e6dbfe18cf8f74717787fd47a7044fda7a3cb81004168ee5"
Oct 30 21:50:47.342656 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:50:47.342643    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ded087dbe4633a08e6dbfe18cf8f74717787fd47a7044fda7a3cb81004168ee5\": container with ID starting with ded087dbe4633a08e6dbfe18cf8f74717787fd47a7044fda7a3cb81004168ee5 not found: ID does not exist" containerID="ded087dbe4633a08e6dbfe18cf8f74717787fd47a7044fda7a3cb81004168ee5"
Oct 30 21:50:47.342698 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.342659    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ded087dbe4633a08e6dbfe18cf8f74717787fd47a7044fda7a3cb81004168ee5"} err="failed to get container status \"ded087dbe4633a08e6dbfe18cf8f74717787fd47a7044fda7a3cb81004168ee5\": rpc error: code = NotFound desc = could not find container \"ded087dbe4633a08e6dbfe18cf8f74717787fd47a7044fda7a3cb81004168ee5\": container with ID starting with ded087dbe4633a08e6dbfe18cf8f74717787fd47a7044fda7a3cb81004168ee5 not found: ID does not exist"
Oct 30 21:50:47.478421 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.478378    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"]
Oct 30 21:50:47.525546 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.525514    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-ffe7a1cf-openshift-default-69875798d5-rcxgg"]
Oct 30 21:50:47.527940 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.527921    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"
Oct 30 21:50:47.559133 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.559094    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e626a30e-ff4e-4e87-9f50-000865cfe6c0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e626a30e-ff4e-4e87-9f50-000865cfe6c0-istio-data\") pod \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\" (UID: \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\") "
Oct 30 21:50:47.559326 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.559157    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e626a30e-ff4e-4e87-9f50-000865cfe6c0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e626a30e-ff4e-4e87-9f50-000865cfe6c0-istiod-ca-cert\") pod \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\" (UID: \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\") "
Oct 30 21:50:47.559326 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.559193    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e626a30e-ff4e-4e87-9f50-000865cfe6c0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e626a30e-ff4e-4e87-9f50-000865cfe6c0-istio-envoy\") pod \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\" (UID: \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\") "
Oct 30 21:50:47.559326 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.559221    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e626a30e-ff4e-4e87-9f50-000865cfe6c0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e626a30e-ff4e-4e87-9f50-000865cfe6c0-istio-podinfo\") pod \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\" (UID: \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\") "
Oct 30 21:50:47.559326 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.559285    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e626a30e-ff4e-4e87-9f50-000865cfe6c0-istio-token\" (UniqueName: \"kubernetes.io/projected/e626a30e-ff4e-4e87-9f50-000865cfe6c0-istio-token\") pod \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\" (UID: \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\") "
Oct 30 21:50:47.559552 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.559327    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e626a30e-ff4e-4e87-9f50-000865cfe6c0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e626a30e-ff4e-4e87-9f50-000865cfe6c0-workload-certs\") pod \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\" (UID: \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\") "
Oct 30 21:50:47.559552 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.559465    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e626a30e-ff4e-4e87-9f50-000865cfe6c0-istio-data" pod "e626a30e-ff4e-4e87-9f50-000865cfe6c0" (UID: "e626a30e-ff4e-4e87-9f50-000865cfe6c0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:47.559552 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.559466    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e626a30e-ff4e-4e87-9f50-000865cfe6c0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e626a30e-ff4e-4e87-9f50-000865cfe6c0-credential-socket\") pod \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\" (UID: \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\") "
Oct 30 21:50:47.559701 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.559554    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e626a30e-ff4e-4e87-9f50-000865cfe6c0-kube-api-access-mbkmg\" (UniqueName: \"kubernetes.io/projected/e626a30e-ff4e-4e87-9f50-000865cfe6c0-kube-api-access-mbkmg\") pod \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\" (UID: \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\") "
Oct 30 21:50:47.559701 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.559613    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e626a30e-ff4e-4e87-9f50-000865cfe6c0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e626a30e-ff4e-4e87-9f50-000865cfe6c0-workload-socket\") pod \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\" (UID: \"e626a30e-ff4e-4e87-9f50-000865cfe6c0\") "
Oct 30 21:50:47.559800 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.559716    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e626a30e-ff4e-4e87-9f50-000865cfe6c0-credential-socket" pod "e626a30e-ff4e-4e87-9f50-000865cfe6c0" (UID: "e626a30e-ff4e-4e87-9f50-000865cfe6c0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:47.559800 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.559725    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e626a30e-ff4e-4e87-9f50-000865cfe6c0-istiod-ca-cert" pod "e626a30e-ff4e-4e87-9f50-000865cfe6c0" (UID: "e626a30e-ff4e-4e87-9f50-000865cfe6c0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:50:47.560022 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.559955    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e626a30e-ff4e-4e87-9f50-000865cfe6c0-workload-certs" pod "e626a30e-ff4e-4e87-9f50-000865cfe6c0" (UID: "e626a30e-ff4e-4e87-9f50-000865cfe6c0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:47.560156 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.559982    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e626a30e-ff4e-4e87-9f50-000865cfe6c0-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:47.560156 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.560070    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e626a30e-ff4e-4e87-9f50-000865cfe6c0-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:47.560156 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.560087    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e626a30e-ff4e-4e87-9f50-000865cfe6c0-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:47.560156 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.560087    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e626a30e-ff4e-4e87-9f50-000865cfe6c0-workload-socket" pod "e626a30e-ff4e-4e87-9f50-000865cfe6c0" (UID: "e626a30e-ff4e-4e87-9f50-000865cfe6c0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:47.561628 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.561599    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e626a30e-ff4e-4e87-9f50-000865cfe6c0-istio-envoy" pod "e626a30e-ff4e-4e87-9f50-000865cfe6c0" (UID: "e626a30e-ff4e-4e87-9f50-000865cfe6c0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:47.561720 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.561625    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e626a30e-ff4e-4e87-9f50-000865cfe6c0-istio-token" pod "e626a30e-ff4e-4e87-9f50-000865cfe6c0" (UID: "e626a30e-ff4e-4e87-9f50-000865cfe6c0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:50:47.561720 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.561641    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e626a30e-ff4e-4e87-9f50-000865cfe6c0-kube-api-access-mbkmg" pod "e626a30e-ff4e-4e87-9f50-000865cfe6c0" (UID: "e626a30e-ff4e-4e87-9f50-000865cfe6c0"). InnerVolumeSpecName "kube-api-access-mbkmg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:50:47.561720 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.561646    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e626a30e-ff4e-4e87-9f50-000865cfe6c0-istio-podinfo" pod "e626a30e-ff4e-4e87-9f50-000865cfe6c0" (UID: "e626a30e-ff4e-4e87-9f50-000865cfe6c0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:50:47.661019 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.660984    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e626a30e-ff4e-4e87-9f50-000865cfe6c0-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:47.661019 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.661017    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e626a30e-ff4e-4e87-9f50-000865cfe6c0-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:47.661250 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.661030    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e626a30e-ff4e-4e87-9f50-000865cfe6c0-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:47.661250 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.661043    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e626a30e-ff4e-4e87-9f50-000865cfe6c0-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:47.661250 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.661055    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbkmg\" (UniqueName: \"kubernetes.io/projected/e626a30e-ff4e-4e87-9f50-000865cfe6c0-kube-api-access-mbkmg\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:47.661250 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.661068    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e626a30e-ff4e-4e87-9f50-000865cfe6c0-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:47.962801 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:47.962765    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="03d16436-58d3-495e-84d4-38b6d3b55a8b" path="/var/lib/kubelet/pods/03d16436-58d3-495e-84d4-38b6d3b55a8b/volumes"
Oct 30 21:50:48.250194 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:48.250119    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-9a5623-597c949cc5-xjcgz"
Oct 30 21:50:48.250194 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:48.250188    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-9a5623-6fbf554f99-kwmhs"
Oct 30 21:50:48.331182 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:48.331146    4871 generic.go:356] "Generic (PLEG): container finished" podID="e626a30e-ff4e-4e87-9f50-000865cfe6c0" containerID="5ce4de346035374e8b7dc73b17db5e80190c1d1c89b9b8e479c2bf6da38cb474" exitCode=0
Oct 30 21:50:48.331751 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:48.331224    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"
Oct 30 21:50:48.331751 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:48.331232    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn" event={"ID":"e626a30e-ff4e-4e87-9f50-000865cfe6c0","Type":"ContainerDied","Data":"5ce4de346035374e8b7dc73b17db5e80190c1d1c89b9b8e479c2bf6da38cb474"}
Oct 30 21:50:48.331751 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:48.331305    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn" event={"ID":"e626a30e-ff4e-4e87-9f50-000865cfe6c0","Type":"ContainerDied","Data":"71f1d5e2e65fe2f9f6bae5449c806da2209d6391d8cedb4a66d7fc4a89cc7d39"}
Oct 30 21:50:48.331751 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:48.331330    4871 scope.go:122] "RemoveContainer" containerID="5ce4de346035374e8b7dc73b17db5e80190c1d1c89b9b8e479c2bf6da38cb474"
Oct 30 21:50:48.341278 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:48.341224    4871 scope.go:122] "RemoveContainer" containerID="94fa926fb003c56e36a68c5a8168d7eafa9f6658ce73dcbfd25763738d50376b"
Oct 30 21:50:48.351424 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:48.351404    4871 scope.go:122] "RemoveContainer" containerID="5ce4de346035374e8b7dc73b17db5e80190c1d1c89b9b8e479c2bf6da38cb474"
Oct 30 21:50:48.351665 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:50:48.351646    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ce4de346035374e8b7dc73b17db5e80190c1d1c89b9b8e479c2bf6da38cb474\": container with ID starting with 5ce4de346035374e8b7dc73b17db5e80190c1d1c89b9b8e479c2bf6da38cb474 not found: ID does not exist" containerID="5ce4de346035374e8b7dc73b17db5e80190c1d1c89b9b8e479c2bf6da38cb474"
Oct 30 21:50:48.351722 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:48.351672    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ce4de346035374e8b7dc73b17db5e80190c1d1c89b9b8e479c2bf6da38cb474"} err="failed to get container status \"5ce4de346035374e8b7dc73b17db5e80190c1d1c89b9b8e479c2bf6da38cb474\": rpc error: code = NotFound desc = could not find container \"5ce4de346035374e8b7dc73b17db5e80190c1d1c89b9b8e479c2bf6da38cb474\": container with ID starting with 5ce4de346035374e8b7dc73b17db5e80190c1d1c89b9b8e479c2bf6da38cb474 not found: ID does not exist"
Oct 30 21:50:48.351722 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:48.351689    4871 scope.go:122] "RemoveContainer" containerID="94fa926fb003c56e36a68c5a8168d7eafa9f6658ce73dcbfd25763738d50376b"
Oct 30 21:50:48.351964 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:50:48.351930    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94fa926fb003c56e36a68c5a8168d7eafa9f6658ce73dcbfd25763738d50376b\": container with ID starting with 94fa926fb003c56e36a68c5a8168d7eafa9f6658ce73dcbfd25763738d50376b not found: ID does not exist" containerID="94fa926fb003c56e36a68c5a8168d7eafa9f6658ce73dcbfd25763738d50376b"
Oct 30 21:50:48.352013 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:48.351974    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94fa926fb003c56e36a68c5a8168d7eafa9f6658ce73dcbfd25763738d50376b"} err="failed to get container status \"94fa926fb003c56e36a68c5a8168d7eafa9f6658ce73dcbfd25763738d50376b\": rpc error: code = NotFound desc = could not find container \"94fa926fb003c56e36a68c5a8168d7eafa9f6658ce73dcbfd25763738d50376b\": container with ID starting with 94fa926fb003c56e36a68c5a8168d7eafa9f6658ce73dcbfd25763738d50376b not found: ID does not exist"
Oct 30 21:50:48.579045 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:48.578957    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"]
Oct 30 21:50:48.585386 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:48.585353    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-16c49820-gw-openshift-default-9d8cdcf57-bvkjn"]
Oct 30 21:50:49.966281 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:49.963404    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e626a30e-ff4e-4e87-9f50-000865cfe6c0" path="/var/lib/kubelet/pods/e626a30e-ff4e-4e87-9f50-000865cfe6c0/volumes"
Oct 30 21:50:50.169186 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:50.169149    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64"]
Oct 30 21:50:50.169591 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:50.169577    4871 cpu_manager.go:414] "containerMap: removing container" podUID="03d16436-58d3-495e-84d4-38b6d3b55a8b" containerName="istio-proxy"
Oct 30 21:50:50.169650 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:50.169593    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="03d16436-58d3-495e-84d4-38b6d3b55a8b" containerName="istio-proxy"
Oct 30 21:50:50.169650 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:50.169619    4871 cpu_manager.go:414] "containerMap: removing container" podUID="03d16436-58d3-495e-84d4-38b6d3b55a8b" containerName="istio-proxy"
Oct 30 21:50:50.169650 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:50.169624    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="03d16436-58d3-495e-84d4-38b6d3b55a8b" containerName="istio-proxy"
Oct 30 21:50:50.169650 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:50.169638    4871 cpu_manager.go:414] "containerMap: removing container" podUID="e626a30e-ff4e-4e87-9f50-000865cfe6c0" containerName="istio-proxy"
Oct 30 21:50:50.169650 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:50.169643    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e626a30e-ff4e-4e87-9f50-000865cfe6c0" containerName="istio-proxy"
Oct 30 21:50:50.169650 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:50.169651    4871 cpu_manager.go:414] "containerMap: removing container" podUID="e626a30e-ff4e-4e87-9f50-000865cfe6c0" containerName="istio-proxy"
Oct 30 21:50:50.169822 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:50.169656    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e626a30e-ff4e-4e87-9f50-000865cfe6c0" containerName="istio-proxy"
Oct 30 21:50:50.169822 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:50.169713    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="03d16436-58d3-495e-84d4-38b6d3b55a8b" containerName="istio-proxy"
Oct 30 21:50:50.169822 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:50.169722    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="e626a30e-ff4e-4e87-9f50-000865cfe6c0" containerName="istio-proxy"
Oct 30 21:50:50.169822 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:50.169728    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="e626a30e-ff4e-4e87-9f50-000865cfe6c0" containerName="istio-proxy"
Oct 30 21:50:50.173016 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:50.172994    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64"
Oct 30 21:50:50.222683 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:50.222602    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 21:50:50.222683 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:50.222603    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-26743a-serving-cert\""
Oct 30 21:50:50.269219 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:50.269184    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 21:50:50.269219 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:50.269184    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 21:50:50.269646 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:50.269258    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-zr4d2\""
Oct 30 21:50:50.272100 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:50.272078    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64"]
Oct 30 21:50:50.287929 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:50.287880    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ac0939d-60c7-400b-b345-4acb5def8620-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64\" (UID: \"3ac0939d-60c7-400b-b345-4acb5def8620\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64"
Oct 30 21:50:50.288085 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:50.287941    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ac0939d-60c7-400b-b345-4acb5def8620-metrics-certs\") pod \"maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64\" (UID: \"3ac0939d-60c7-400b-b345-4acb5def8620\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64"
Oct 30 21:50:50.288085 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:50.287972    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ftsq\" (UniqueName: \"kubernetes.io/projected/3ac0939d-60c7-400b-b345-4acb5def8620-kube-api-access-5ftsq\") pod \"maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64\" (UID: \"3ac0939d-60c7-400b-b345-4acb5def8620\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64"
Oct 30 21:50:50.288085 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:50.288074    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ac0939d-60c7-400b-b345-4acb5def8620-maas-api-tls\") pod \"maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64\" (UID: \"3ac0939d-60c7-400b-b345-4acb5def8620\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64"
Oct 30 21:50:50.359383 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:50.359352    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:50.388694 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:50.388650    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ac0939d-60c7-400b-b345-4acb5def8620-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64\" (UID: \"3ac0939d-60c7-400b-b345-4acb5def8620\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64"
Oct 30 21:50:50.388875 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:50.388712    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ac0939d-60c7-400b-b345-4acb5def8620-metrics-certs\") pod \"maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64\" (UID: \"3ac0939d-60c7-400b-b345-4acb5def8620\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64"
Oct 30 21:50:50.388875 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:50.388763    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ftsq\" (UniqueName: \"kubernetes.io/projected/3ac0939d-60c7-400b-b345-4acb5def8620-kube-api-access-5ftsq\") pod \"maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64\" (UID: \"3ac0939d-60c7-400b-b345-4acb5def8620\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64"
Oct 30 21:50:50.388875 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:50.388855    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ac0939d-60c7-400b-b345-4acb5def8620-maas-api-tls\") pod \"maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64\" (UID: \"3ac0939d-60c7-400b-b345-4acb5def8620\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64"
Oct 30 21:50:50.389053 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:50:50.389005    4871 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-26743a-serving-cert: secret "maas-api-e2e-worker-w4-26743a-serving-cert" not found
Oct 30 21:50:50.389114 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:50:50.389081    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ac0939d-60c7-400b-b345-4acb5def8620-maas-api-tls podName:3ac0939d-60c7-400b-b345-4acb5def8620 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:50.889056306 +0000 UTC m=+793.066544906 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3ac0939d-60c7-400b-b345-4acb5def8620-maas-api-tls") pod "maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64" (UID: "3ac0939d-60c7-400b-b345-4acb5def8620") : secret "maas-api-e2e-worker-w4-26743a-serving-cert" not found
Oct 30 21:50:50.389346 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:50.389321    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ac0939d-60c7-400b-b345-4acb5def8620-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64\" (UID: \"3ac0939d-60c7-400b-b345-4acb5def8620\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64"
Oct 30 21:50:50.391177 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:50.391154    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ac0939d-60c7-400b-b345-4acb5def8620-metrics-certs\") pod \"maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64\" (UID: \"3ac0939d-60c7-400b-b345-4acb5def8620\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64"
Oct 30 21:50:50.447877 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:50.447846    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ftsq\" (UniqueName: \"kubernetes.io/projected/3ac0939d-60c7-400b-b345-4acb5def8620-kube-api-access-5ftsq\") pod \"maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64\" (UID: \"3ac0939d-60c7-400b-b345-4acb5def8620\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64"
Oct 30 21:50:50.512716 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:50.512595    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:50.667884 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:50.667832    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:50.894596 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:50.894556    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ac0939d-60c7-400b-b345-4acb5def8620-maas-api-tls\") pod \"maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64\" (UID: \"3ac0939d-60c7-400b-b345-4acb5def8620\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64"
Oct 30 21:50:50.897130 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:50.897101    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ac0939d-60c7-400b-b345-4acb5def8620-maas-api-tls\") pod \"maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64\" (UID: \"3ac0939d-60c7-400b-b345-4acb5def8620\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64"
Oct 30 21:50:51.085292 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.085227    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64"
Oct 30 21:50:51.284659 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.284623    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"]
Oct 30 21:50:51.285233 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.285215    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="03d16436-58d3-495e-84d4-38b6d3b55a8b" containerName="istio-proxy"
Oct 30 21:50:51.289915 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.289893    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"
Oct 30 21:50:51.352344 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.352309    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-4d14e0-openshift-default-dockercfg-mqz9f\""
Oct 30 21:50:51.398362 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.398231    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1969a36c-2ac9-4972-8a9e-e84e39682a28-workload-socket\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx\" (UID: \"1969a36c-2ac9-4972-8a9e-e84e39682a28\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"
Oct 30 21:50:51.398552 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.398424    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1969a36c-2ac9-4972-8a9e-e84e39682a28-istio-envoy\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx\" (UID: \"1969a36c-2ac9-4972-8a9e-e84e39682a28\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"
Oct 30 21:50:51.398552 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.398486    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1969a36c-2ac9-4972-8a9e-e84e39682a28-istio-data\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx\" (UID: \"1969a36c-2ac9-4972-8a9e-e84e39682a28\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"
Oct 30 21:50:51.398552 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.398519    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1969a36c-2ac9-4972-8a9e-e84e39682a28-credential-socket\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx\" (UID: \"1969a36c-2ac9-4972-8a9e-e84e39682a28\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"
Oct 30 21:50:51.398726 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.398545    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1969a36c-2ac9-4972-8a9e-e84e39682a28-istiod-ca-cert\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx\" (UID: \"1969a36c-2ac9-4972-8a9e-e84e39682a28\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"
Oct 30 21:50:51.398726 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.398598    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1969a36c-2ac9-4972-8a9e-e84e39682a28-istio-token\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx\" (UID: \"1969a36c-2ac9-4972-8a9e-e84e39682a28\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"
Oct 30 21:50:51.399831 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.399800    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bblw\" (UniqueName: \"kubernetes.io/projected/1969a36c-2ac9-4972-8a9e-e84e39682a28-kube-api-access-4bblw\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx\" (UID: \"1969a36c-2ac9-4972-8a9e-e84e39682a28\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"
Oct 30 21:50:51.399966 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.399930    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1969a36c-2ac9-4972-8a9e-e84e39682a28-workload-certs\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx\" (UID: \"1969a36c-2ac9-4972-8a9e-e84e39682a28\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"
Oct 30 21:50:51.400014 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.399967    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1969a36c-2ac9-4972-8a9e-e84e39682a28-istio-podinfo\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx\" (UID: \"1969a36c-2ac9-4972-8a9e-e84e39682a28\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"
Oct 30 21:50:51.401275 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.401224    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:51.401982 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.401959    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"]
Oct 30 21:50:51.500603 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.500571    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1969a36c-2ac9-4972-8a9e-e84e39682a28-istio-data\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx\" (UID: \"1969a36c-2ac9-4972-8a9e-e84e39682a28\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"
Oct 30 21:50:51.500603 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.500607    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1969a36c-2ac9-4972-8a9e-e84e39682a28-credential-socket\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx\" (UID: \"1969a36c-2ac9-4972-8a9e-e84e39682a28\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"
Oct 30 21:50:51.500867 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.500627    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1969a36c-2ac9-4972-8a9e-e84e39682a28-istiod-ca-cert\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx\" (UID: \"1969a36c-2ac9-4972-8a9e-e84e39682a28\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"
Oct 30 21:50:51.500867 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.500762    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1969a36c-2ac9-4972-8a9e-e84e39682a28-istio-token\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx\" (UID: \"1969a36c-2ac9-4972-8a9e-e84e39682a28\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"
Oct 30 21:50:51.500867 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.500807    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bblw\" (UniqueName: \"kubernetes.io/projected/1969a36c-2ac9-4972-8a9e-e84e39682a28-kube-api-access-4bblw\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx\" (UID: \"1969a36c-2ac9-4972-8a9e-e84e39682a28\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"
Oct 30 21:50:51.500867 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.500852    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1969a36c-2ac9-4972-8a9e-e84e39682a28-workload-certs\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx\" (UID: \"1969a36c-2ac9-4972-8a9e-e84e39682a28\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"
Oct 30 21:50:51.501082 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.500889    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1969a36c-2ac9-4972-8a9e-e84e39682a28-istio-podinfo\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx\" (UID: \"1969a36c-2ac9-4972-8a9e-e84e39682a28\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"
Oct 30 21:50:51.501082 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.500956    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1969a36c-2ac9-4972-8a9e-e84e39682a28-workload-socket\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx\" (UID: \"1969a36c-2ac9-4972-8a9e-e84e39682a28\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"
Oct 30 21:50:51.501082 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.501028    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1969a36c-2ac9-4972-8a9e-e84e39682a28-istio-envoy\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx\" (UID: \"1969a36c-2ac9-4972-8a9e-e84e39682a28\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"
Oct 30 21:50:51.501082 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.501062    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1969a36c-2ac9-4972-8a9e-e84e39682a28-credential-socket\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx\" (UID: \"1969a36c-2ac9-4972-8a9e-e84e39682a28\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"
Oct 30 21:50:51.501383 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.501360    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1969a36c-2ac9-4972-8a9e-e84e39682a28-workload-certs\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx\" (UID: \"1969a36c-2ac9-4972-8a9e-e84e39682a28\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"
Oct 30 21:50:51.501431 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.501410    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1969a36c-2ac9-4972-8a9e-e84e39682a28-istiod-ca-cert\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx\" (UID: \"1969a36c-2ac9-4972-8a9e-e84e39682a28\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"
Oct 30 21:50:51.501465 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.501450    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1969a36c-2ac9-4972-8a9e-e84e39682a28-workload-socket\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx\" (UID: \"1969a36c-2ac9-4972-8a9e-e84e39682a28\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"
Oct 30 21:50:51.501583 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.501565    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1969a36c-2ac9-4972-8a9e-e84e39682a28-istio-data\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx\" (UID: \"1969a36c-2ac9-4972-8a9e-e84e39682a28\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"
Oct 30 21:50:51.503681 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.503652    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1969a36c-2ac9-4972-8a9e-e84e39682a28-istio-envoy\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx\" (UID: \"1969a36c-2ac9-4972-8a9e-e84e39682a28\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"
Oct 30 21:50:51.503850 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.503832    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1969a36c-2ac9-4972-8a9e-e84e39682a28-istio-podinfo\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx\" (UID: \"1969a36c-2ac9-4972-8a9e-e84e39682a28\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"
Oct 30 21:50:51.505928 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.505908    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64"]
Oct 30 21:50:51.507683 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:50:51.507661    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ac0939d_60c7_400b_b345_4acb5def8620.slice/crio-2e693baa099c8228d686e791146deb0df856f026c446adf37d1f44294fa4c210 WatchSource:0}: Error finding container 2e693baa099c8228d686e791146deb0df856f026c446adf37d1f44294fa4c210: Status 404 returned error can't find the container with id 2e693baa099c8228d686e791146deb0df856f026c446adf37d1f44294fa4c210
Oct 30 21:50:51.556778 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.556732    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:51.558973 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.558950    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1969a36c-2ac9-4972-8a9e-e84e39682a28-istio-token\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx\" (UID: \"1969a36c-2ac9-4972-8a9e-e84e39682a28\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"
Oct 30 21:50:51.559155 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.559135    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bblw\" (UniqueName: \"kubernetes.io/projected/1969a36c-2ac9-4972-8a9e-e84e39682a28-kube-api-access-4bblw\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx\" (UID: \"1969a36c-2ac9-4972-8a9e-e84e39682a28\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"
Oct 30 21:50:51.607362 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.607158    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"
Oct 30 21:50:51.709361 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.709323    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:51.924440 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.924406    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-56f3fe-6689c845d7-c2f5w"]
Oct 30 21:50:51.929345 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.929278    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-56f3fe-6689c845d7-c2f5w"
Oct 30 21:50:51.979755 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.979697    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w2-56f3fe\""
Oct 30 21:50:51.979999 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.979858    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w2-56f3fe-dockercfg-bqdzl\""
Oct 30 21:50:51.983105 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:51.983073    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-56f3fe-6689c845d7-c2f5w"]
Oct 30 21:50:52.031260 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:52.026848    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfs6b\" (UniqueName: \"kubernetes.io/projected/5a47bd06-49c0-45df-a003-78ab3c1e2d22-kube-api-access-zfs6b\") pod \"payload-pre-processing-e2e-worker-w2-56f3fe-6689c845d7-c2f5w\" (UID: \"5a47bd06-49c0-45df-a003-78ab3c1e2d22\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-56f3fe-6689c845d7-c2f5w"
Oct 30 21:50:52.031260 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:52.026934    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a47bd06-49c0-45df-a003-78ab3c1e2d22-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-56f3fe-6689c845d7-c2f5w\" (UID: \"5a47bd06-49c0-45df-a003-78ab3c1e2d22\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-56f3fe-6689c845d7-c2f5w"
Oct 30 21:50:52.048259 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:52.048178    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"]
Oct 30 21:50:52.062766 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:52.062726    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:50:52.062894 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:52.062811    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:50:52.062894 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:52.062853    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:50:52.062894 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:52.062869    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-56f3fe-78865dc69c-22kpq"]
Oct 30 21:50:52.066921 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:52.066900    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-56f3fe-78865dc69c-22kpq"
Oct 30 21:50:52.119797 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:52.119767    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-56f3fe-78865dc69c-22kpq"]
Oct 30 21:50:52.128345 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:52.128313    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/345d46dc-dc95-43e6-beaf-8615bfa7d148-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-56f3fe-78865dc69c-22kpq\" (UID: \"345d46dc-dc95-43e6-beaf-8615bfa7d148\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-56f3fe-78865dc69c-22kpq"
Oct 30 21:50:52.128542 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:52.128454    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zfm8\" (UniqueName: \"kubernetes.io/projected/345d46dc-dc95-43e6-beaf-8615bfa7d148-kube-api-access-2zfm8\") pod \"payload-processing-e2e-worker-w2-56f3fe-78865dc69c-22kpq\" (UID: \"345d46dc-dc95-43e6-beaf-8615bfa7d148\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-56f3fe-78865dc69c-22kpq"
Oct 30 21:50:52.128542 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:52.128528    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfs6b\" (UniqueName: \"kubernetes.io/projected/5a47bd06-49c0-45df-a003-78ab3c1e2d22-kube-api-access-zfs6b\") pod \"payload-pre-processing-e2e-worker-w2-56f3fe-6689c845d7-c2f5w\" (UID: \"5a47bd06-49c0-45df-a003-78ab3c1e2d22\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-56f3fe-6689c845d7-c2f5w"
Oct 30 21:50:52.128679 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:52.128594    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a47bd06-49c0-45df-a003-78ab3c1e2d22-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-56f3fe-6689c845d7-c2f5w\" (UID: \"5a47bd06-49c0-45df-a003-78ab3c1e2d22\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-56f3fe-6689c845d7-c2f5w"
Oct 30 21:50:52.129542 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:52.129515    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a47bd06-49c0-45df-a003-78ab3c1e2d22-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-56f3fe-6689c845d7-c2f5w\" (UID: \"5a47bd06-49c0-45df-a003-78ab3c1e2d22\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-56f3fe-6689c845d7-c2f5w"
Oct 30 21:50:52.186704 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:52.186623    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfs6b\" (UniqueName: \"kubernetes.io/projected/5a47bd06-49c0-45df-a003-78ab3c1e2d22-kube-api-access-zfs6b\") pod \"payload-pre-processing-e2e-worker-w2-56f3fe-6689c845d7-c2f5w\" (UID: \"5a47bd06-49c0-45df-a003-78ab3c1e2d22\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-56f3fe-6689c845d7-c2f5w"
Oct 30 21:50:52.229740 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:52.229691    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zfm8\" (UniqueName: \"kubernetes.io/projected/345d46dc-dc95-43e6-beaf-8615bfa7d148-kube-api-access-2zfm8\") pod \"payload-processing-e2e-worker-w2-56f3fe-78865dc69c-22kpq\" (UID: \"345d46dc-dc95-43e6-beaf-8615bfa7d148\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-56f3fe-78865dc69c-22kpq"
Oct 30 21:50:52.229944 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:52.229848    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/345d46dc-dc95-43e6-beaf-8615bfa7d148-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-56f3fe-78865dc69c-22kpq\" (UID: \"345d46dc-dc95-43e6-beaf-8615bfa7d148\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-56f3fe-78865dc69c-22kpq"
Oct 30 21:50:52.230729 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:52.230710    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/345d46dc-dc95-43e6-beaf-8615bfa7d148-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-56f3fe-78865dc69c-22kpq\" (UID: \"345d46dc-dc95-43e6-beaf-8615bfa7d148\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-56f3fe-78865dc69c-22kpq"
Oct 30 21:50:52.243631 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:52.243597    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-56f3fe-6689c845d7-c2f5w"
Oct 30 21:50:52.289179 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:52.289147    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zfm8\" (UniqueName: \"kubernetes.io/projected/345d46dc-dc95-43e6-beaf-8615bfa7d148-kube-api-access-2zfm8\") pod \"payload-processing-e2e-worker-w2-56f3fe-78865dc69c-22kpq\" (UID: \"345d46dc-dc95-43e6-beaf-8615bfa7d148\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-56f3fe-78865dc69c-22kpq"
Oct 30 21:50:52.373478 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:52.373437    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx" event={"ID":"1969a36c-2ac9-4972-8a9e-e84e39682a28","Type":"ContainerStarted","Data":"cb13b4186f0a942b24f7f268d5138b480ab6df06de9ac90c33879029c3618849"}
Oct 30 21:50:52.373686 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:52.373504    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx" event={"ID":"1969a36c-2ac9-4972-8a9e-e84e39682a28","Type":"ContainerStarted","Data":"3a18e13917f1ef78563593db17981e4deaaf32825130a0051e15c0a76c8a8d08"}
Oct 30 21:50:52.374792 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:52.374765    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64" event={"ID":"3ac0939d-60c7-400b-b345-4acb5def8620","Type":"ContainerStarted","Data":"2e693baa099c8228d686e791146deb0df856f026c446adf37d1f44294fa4c210"}
Oct 30 21:50:52.381022 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:52.380994    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-56f3fe-78865dc69c-22kpq"
Oct 30 21:50:52.492540 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:52.492480    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx" podStartSLOduration=1.492463011 podStartE2EDuration="1.492463011s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:50:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:50:52.490310943 +0000 UTC m=+794.667799560" watchObservedRunningTime="2026-10-30 21:50:52.492463011 +0000 UTC m=+794.669951627"
Oct 30 21:50:52.546570 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:52.546533    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:52.613259 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:52.612333    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"
Oct 30 21:50:52.618272 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:52.618200    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-56f3fe-6689c845d7-c2f5w"]
Oct 30 21:50:52.632332 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:50:52.630451    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a47bd06_49c0_45df_a003_78ab3c1e2d22.slice/crio-f6786a4eda806742b63c25236fbbdc7bfe803373b6e30833e0a793973372ba76 WatchSource:0}: Error finding container f6786a4eda806742b63c25236fbbdc7bfe803373b6e30833e0a793973372ba76: Status 404 returned error can't find the container with id f6786a4eda806742b63c25236fbbdc7bfe803373b6e30833e0a793973372ba76
Oct 30 21:50:52.761387 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:52.761360    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-56f3fe-78865dc69c-22kpq"]
Oct 30 21:50:52.763535 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:50:52.763506    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod345d46dc_dc95_43e6_beaf_8615bfa7d148.slice/crio-7fb08e3a8b03c15fa485aa04ba4975f1999fd9d3a00b72e34282cc78411b51de WatchSource:0}: Error finding container 7fb08e3a8b03c15fa485aa04ba4975f1999fd9d3a00b72e34282cc78411b51de: Status 404 returned error can't find the container with id 7fb08e3a8b03c15fa485aa04ba4975f1999fd9d3a00b72e34282cc78411b51de
Oct 30 21:50:53.077072 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.076994    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"]
Oct 30 21:50:53.080722 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.080696    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"
Oct 30 21:50:53.151263 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.135479    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-3c4802-openshift-default-dockercfg-6gcqv\""
Oct 30 21:50:53.181001 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.180881    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:53.182614 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.182584    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"]
Oct 30 21:50:53.263813 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.263769    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3798606-a072-44a5-891c-4588de3c6ff1-istiod-ca-cert\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm\" (UID: \"f3798606-a072-44a5-891c-4588de3c6ff1\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"
Oct 30 21:50:53.264012 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.263821    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxhnc\" (UniqueName: \"kubernetes.io/projected/f3798606-a072-44a5-891c-4588de3c6ff1-kube-api-access-sxhnc\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm\" (UID: \"f3798606-a072-44a5-891c-4588de3c6ff1\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"
Oct 30 21:50:53.264012 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.263862    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3798606-a072-44a5-891c-4588de3c6ff1-workload-socket\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm\" (UID: \"f3798606-a072-44a5-891c-4588de3c6ff1\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"
Oct 30 21:50:53.264012 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.263902    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3798606-a072-44a5-891c-4588de3c6ff1-istio-data\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm\" (UID: \"f3798606-a072-44a5-891c-4588de3c6ff1\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"
Oct 30 21:50:53.264012 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.263924    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3798606-a072-44a5-891c-4588de3c6ff1-istio-token\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm\" (UID: \"f3798606-a072-44a5-891c-4588de3c6ff1\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"
Oct 30 21:50:53.264012 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.263947    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3798606-a072-44a5-891c-4588de3c6ff1-workload-certs\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm\" (UID: \"f3798606-a072-44a5-891c-4588de3c6ff1\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"
Oct 30 21:50:53.264012 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.263984    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3798606-a072-44a5-891c-4588de3c6ff1-credential-socket\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm\" (UID: \"f3798606-a072-44a5-891c-4588de3c6ff1\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"
Oct 30 21:50:53.264012 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.264000    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3798606-a072-44a5-891c-4588de3c6ff1-istio-podinfo\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm\" (UID: \"f3798606-a072-44a5-891c-4588de3c6ff1\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"
Oct 30 21:50:53.264393 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.264050    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3798606-a072-44a5-891c-4588de3c6ff1-istio-envoy\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm\" (UID: \"f3798606-a072-44a5-891c-4588de3c6ff1\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"
Oct 30 21:50:53.364952 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.364910    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxhnc\" (UniqueName: \"kubernetes.io/projected/f3798606-a072-44a5-891c-4588de3c6ff1-kube-api-access-sxhnc\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm\" (UID: \"f3798606-a072-44a5-891c-4588de3c6ff1\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"
Oct 30 21:50:53.365165 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.364978    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3798606-a072-44a5-891c-4588de3c6ff1-workload-socket\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm\" (UID: \"f3798606-a072-44a5-891c-4588de3c6ff1\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"
Oct 30 21:50:53.365165 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.365023    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3798606-a072-44a5-891c-4588de3c6ff1-istio-data\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm\" (UID: \"f3798606-a072-44a5-891c-4588de3c6ff1\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"
Oct 30 21:50:53.365165 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.365041    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3798606-a072-44a5-891c-4588de3c6ff1-istio-token\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm\" (UID: \"f3798606-a072-44a5-891c-4588de3c6ff1\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"
Oct 30 21:50:53.365165 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.365063    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3798606-a072-44a5-891c-4588de3c6ff1-workload-certs\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm\" (UID: \"f3798606-a072-44a5-891c-4588de3c6ff1\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"
Oct 30 21:50:53.365165 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.365098    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3798606-a072-44a5-891c-4588de3c6ff1-credential-socket\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm\" (UID: \"f3798606-a072-44a5-891c-4588de3c6ff1\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"
Oct 30 21:50:53.365165 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.365115    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3798606-a072-44a5-891c-4588de3c6ff1-istio-podinfo\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm\" (UID: \"f3798606-a072-44a5-891c-4588de3c6ff1\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"
Oct 30 21:50:53.365550 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.365173    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3798606-a072-44a5-891c-4588de3c6ff1-istio-envoy\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm\" (UID: \"f3798606-a072-44a5-891c-4588de3c6ff1\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"
Oct 30 21:50:53.365550 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.365226    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3798606-a072-44a5-891c-4588de3c6ff1-istiod-ca-cert\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm\" (UID: \"f3798606-a072-44a5-891c-4588de3c6ff1\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"
Oct 30 21:50:53.365782 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.365708    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3798606-a072-44a5-891c-4588de3c6ff1-workload-certs\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm\" (UID: \"f3798606-a072-44a5-891c-4588de3c6ff1\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"
Oct 30 21:50:53.366367 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.366027    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3798606-a072-44a5-891c-4588de3c6ff1-istio-data\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm\" (UID: \"f3798606-a072-44a5-891c-4588de3c6ff1\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"
Oct 30 21:50:53.366367 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.366076    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3798606-a072-44a5-891c-4588de3c6ff1-istiod-ca-cert\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm\" (UID: \"f3798606-a072-44a5-891c-4588de3c6ff1\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"
Oct 30 21:50:53.366367 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.366083    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3798606-a072-44a5-891c-4588de3c6ff1-workload-socket\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm\" (UID: \"f3798606-a072-44a5-891c-4588de3c6ff1\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"
Oct 30 21:50:53.366367 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.366321    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3798606-a072-44a5-891c-4588de3c6ff1-credential-socket\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm\" (UID: \"f3798606-a072-44a5-891c-4588de3c6ff1\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"
Oct 30 21:50:53.368905 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.368883    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3798606-a072-44a5-891c-4588de3c6ff1-istio-podinfo\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm\" (UID: \"f3798606-a072-44a5-891c-4588de3c6ff1\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"
Oct 30 21:50:53.369015 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.368995    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3798606-a072-44a5-891c-4588de3c6ff1-istio-envoy\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm\" (UID: \"f3798606-a072-44a5-891c-4588de3c6ff1\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"
Oct 30 21:50:53.386197 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.386157    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-56f3fe-6689c845d7-c2f5w" event={"ID":"5a47bd06-49c0-45df-a003-78ab3c1e2d22","Type":"ContainerStarted","Data":"1d175e0aa4dd8dbc0e7bb5d6eac8b0d30643d3605bd3072c44a2b8d2c00816cc"}
Oct 30 21:50:53.386432 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.386256    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-56f3fe-6689c845d7-c2f5w"
Oct 30 21:50:53.386432 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.386287    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-56f3fe-6689c845d7-c2f5w" event={"ID":"5a47bd06-49c0-45df-a003-78ab3c1e2d22","Type":"ContainerStarted","Data":"f6786a4eda806742b63c25236fbbdc7bfe803373b6e30833e0a793973372ba76"}
Oct 30 21:50:53.390437 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.390401    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-56f3fe-78865dc69c-22kpq" event={"ID":"345d46dc-dc95-43e6-beaf-8615bfa7d148","Type":"ContainerStarted","Data":"bdb92f0d739899e9fa9044f573bca7683e2fc8964501719cebfb64e5c1b0549b"}
Oct 30 21:50:53.390591 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.390465    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-56f3fe-78865dc69c-22kpq"
Oct 30 21:50:53.390591 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.390490    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-56f3fe-78865dc69c-22kpq" event={"ID":"345d46dc-dc95-43e6-beaf-8615bfa7d148","Type":"ContainerStarted","Data":"7fb08e3a8b03c15fa485aa04ba4975f1999fd9d3a00b72e34282cc78411b51de"}
Oct 30 21:50:53.424486 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.424446    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxhnc\" (UniqueName: \"kubernetes.io/projected/f3798606-a072-44a5-891c-4588de3c6ff1-kube-api-access-sxhnc\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm\" (UID: \"f3798606-a072-44a5-891c-4588de3c6ff1\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"
Oct 30 21:50:53.425854 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.425810    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3798606-a072-44a5-891c-4588de3c6ff1-istio-token\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm\" (UID: \"f3798606-a072-44a5-891c-4588de3c6ff1\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"
Oct 30 21:50:53.453934 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.453897    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:53.613874 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.613827    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"
Oct 30 21:50:53.624755 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.619021    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w2-56f3fe-78865dc69c-22kpq" podStartSLOduration=1.619000705 podStartE2EDuration="1.619000705s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:50:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:50:53.616593519 +0000 UTC m=+795.794082139" watchObservedRunningTime="2026-10-30 21:50:53.619000705 +0000 UTC m=+795.796489323"
Oct 30 21:50:53.624755 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.620222    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-56f3fe-6689c845d7-c2f5w" podStartSLOduration=2.620207219 podStartE2EDuration="2.620207219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:50:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:50:53.506995818 +0000 UTC m=+795.684484433" watchObservedRunningTime="2026-10-30 21:50:53.620207219 +0000 UTC m=+795.797695838"
Oct 30 21:50:53.696023 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:53.695925    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"
Oct 30 21:50:54.155524 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:54.155475    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"]
Oct 30 21:50:54.157003 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:50:54.156957    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3798606_a072_44a5_891c_4588de3c6ff1.slice/crio-8c5aff5aa9bc6716895b2434407ac7568f5b438616c51bf3dd427e059edf63e6 WatchSource:0}: Error finding container 8c5aff5aa9bc6716895b2434407ac7568f5b438616c51bf3dd427e059edf63e6: Status 404 returned error can't find the container with id 8c5aff5aa9bc6716895b2434407ac7568f5b438616c51bf3dd427e059edf63e6
Oct 30 21:50:54.160225 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:54.159737    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:50:54.160225 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:54.159802    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:50:54.160225 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:54.159832    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:50:54.399060 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:54.399023    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm" event={"ID":"f3798606-a072-44a5-891c-4588de3c6ff1","Type":"ContainerStarted","Data":"32f8069ac79b7a359568aa080a026b48d32973596ce134cef15097cb136fcddf"}
Oct 30 21:50:54.399251 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:54.399082    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm" event={"ID":"f3798606-a072-44a5-891c-4588de3c6ff1","Type":"ContainerStarted","Data":"8c5aff5aa9bc6716895b2434407ac7568f5b438616c51bf3dd427e059edf63e6"}
Oct 30 21:50:54.400884 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:54.400848    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64" event={"ID":"3ac0939d-60c7-400b-b345-4acb5def8620","Type":"ContainerStarted","Data":"a247084742c0a20660f4cecd48d74eb2fa392d43647679d6aa8500fd0600d9cc"}
Oct 30 21:50:54.401356 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:54.401331    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"
Oct 30 21:50:54.402223 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:54.402202    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"
Oct 30 21:50:54.516915 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:54.516788    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm" podStartSLOduration=1.516767839 podStartE2EDuration="1.516767839s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:50:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:50:54.514675651 +0000 UTC m=+796.692164267" watchObservedRunningTime="2026-10-30 21:50:54.516767839 +0000 UTC m=+796.694256457"
Oct 30 21:50:54.628612 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:54.628549    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64" podStartSLOduration=2.429131196 podStartE2EDuration="4.628533007s" totalImagesPullingTime="2.199401811s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:50:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:50:54.626534523 +0000 UTC m=+796.804023141" watchObservedRunningTime="2026-10-30 21:50:54.628533007 +0000 UTC m=+796.806021623"
Oct 30 21:50:54.696887 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:54.696842    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"
Oct 30 21:50:55.298399 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:55.298354    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:55.408207 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:55.408176    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64"
Oct 30 21:50:55.459222 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:55.459162    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:55.607862 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:55.607769    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:55.702929 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:55.702900    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"
Oct 30 21:50:56.281932 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:56.281892    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:56.423280 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:56.418722    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"
Oct 30 21:50:56.423280 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:56.420400    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"
Oct 30 21:50:56.443357 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:56.443323    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:56.632332 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:56.632294    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:57.325701 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:57.325673    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-26743a-75976bb55f-5vl8h"
Oct 30 21:50:57.325868 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:57.325749    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-26743a-fcb59c9c7-s4rnl"
Oct 30 21:50:58.893108 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:58.893065    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:59.169833 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:59.169744    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:59.376095 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:50:59.376050    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:00.447155 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:00.447117    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:01.424151 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:01.424123    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64"
Oct 30 21:51:01.461614 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:01.461578    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:01.643701 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:01.643651    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:02.021125 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:02.021086    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-4d14e0-8577b8cdc7-tpcg9"]
Oct 30 21:51:02.026232 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:02.026204    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-4d14e0-8577b8cdc7-tpcg9"
Oct 30 21:51:02.067841 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:02.067807    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbtqg\" (UniqueName: \"kubernetes.io/projected/d9b034ee-198d-4fcc-b726-78f720f65468-kube-api-access-jbtqg\") pod \"payload-processing-e2e-worker-w3-4d14e0-8577b8cdc7-tpcg9\" (UID: \"d9b034ee-198d-4fcc-b726-78f720f65468\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-4d14e0-8577b8cdc7-tpcg9"
Oct 30 21:51:02.068030 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:02.067856    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9b034ee-198d-4fcc-b726-78f720f65468-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-4d14e0-8577b8cdc7-tpcg9\" (UID: \"d9b034ee-198d-4fcc-b726-78f720f65468\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-4d14e0-8577b8cdc7-tpcg9"
Oct 30 21:51:02.076106 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:02.076074    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-4d14e0\""
Oct 30 21:51:02.076277 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:02.076108    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-4d14e0-dockercfg-vw9vx\""
Oct 30 21:51:02.125397 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:02.125355    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-4d14e0-8577b8cdc7-tpcg9"]
Oct 30 21:51:02.169043 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:02.169012    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbtqg\" (UniqueName: \"kubernetes.io/projected/d9b034ee-198d-4fcc-b726-78f720f65468-kube-api-access-jbtqg\") pod \"payload-processing-e2e-worker-w3-4d14e0-8577b8cdc7-tpcg9\" (UID: \"d9b034ee-198d-4fcc-b726-78f720f65468\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-4d14e0-8577b8cdc7-tpcg9"
Oct 30 21:51:02.169201 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:02.169055    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9b034ee-198d-4fcc-b726-78f720f65468-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-4d14e0-8577b8cdc7-tpcg9\" (UID: \"d9b034ee-198d-4fcc-b726-78f720f65468\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-4d14e0-8577b8cdc7-tpcg9"
Oct 30 21:51:02.169810 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:02.169792    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9b034ee-198d-4fcc-b726-78f720f65468-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-4d14e0-8577b8cdc7-tpcg9\" (UID: \"d9b034ee-198d-4fcc-b726-78f720f65468\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-4d14e0-8577b8cdc7-tpcg9"
Oct 30 21:51:02.230644 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:02.230606    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbtqg\" (UniqueName: \"kubernetes.io/projected/d9b034ee-198d-4fcc-b726-78f720f65468-kube-api-access-jbtqg\") pod \"payload-processing-e2e-worker-w3-4d14e0-8577b8cdc7-tpcg9\" (UID: \"d9b034ee-198d-4fcc-b726-78f720f65468\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-4d14e0-8577b8cdc7-tpcg9"
Oct 30 21:51:02.340866 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:02.340764    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-4d14e0-8577b8cdc7-tpcg9"
Oct 30 21:51:02.716576 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:02.716543    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-4d14e0-8577b8cdc7-tpcg9"]
Oct 30 21:51:02.718383 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:51:02.718342    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9b034ee_198d_4fcc_b726_78f720f65468.slice/crio-ba0f395d60d417b2c3a90a2d8dd648e82f01702de6acdc25248e735fc6bd4ca1 WatchSource:0}: Error finding container ba0f395d60d417b2c3a90a2d8dd648e82f01702de6acdc25248e735fc6bd4ca1: Status 404 returned error can't find the container with id ba0f395d60d417b2c3a90a2d8dd648e82f01702de6acdc25248e735fc6bd4ca1
Oct 30 21:51:03.471479 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:03.471438    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-4d14e0-8577b8cdc7-tpcg9" event={"ID":"d9b034ee-198d-4fcc-b726-78f720f65468","Type":"ContainerStarted","Data":"e2866eb72cffce234c711621529db1ebe552e1ac1a63a546751a9b178a719d6b"}
Oct 30 21:51:03.471678 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:03.471518    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-4d14e0-8577b8cdc7-tpcg9" event={"ID":"d9b034ee-198d-4fcc-b726-78f720f65468","Type":"ContainerStarted","Data":"ba0f395d60d417b2c3a90a2d8dd648e82f01702de6acdc25248e735fc6bd4ca1"}
Oct 30 21:51:03.471678 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:03.471557    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-4d14e0-8577b8cdc7-tpcg9"
Oct 30 21:51:03.584084 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:03.584019    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w3-4d14e0-8577b8cdc7-tpcg9" podStartSLOduration=2.584003667 podStartE2EDuration="2.584003667s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:51:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:51:03.581304709 +0000 UTC m=+805.758793325" watchObservedRunningTime="2026-10-30 21:51:03.584003667 +0000 UTC m=+805.761492283"
Oct 30 21:51:04.404092 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:04.404057    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-56f3fe-78865dc69c-22kpq"
Oct 30 21:51:04.404496 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:04.404286    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-56f3fe-6689c845d7-c2f5w"
Oct 30 21:51:05.532491 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:05.532461    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:05.571711 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:51:05.571665    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:51:09.006479 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:09.006438    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:09.195562 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:09.195529    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:09.426172 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:09.426135    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-3c4802-7bd9747b5f-sfrmg"]
Oct 30 21:51:09.430452 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:09.430430    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-3c4802-7bd9747b5f-sfrmg"
Oct 30 21:51:09.484588 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:09.484545    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-3c4802-7bd9747b5f-sfrmg"]
Oct 30 21:51:09.527925 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:09.527892    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w1-3c4802\""
Oct 30 21:51:09.528164 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:09.527892    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w1-3c4802-dockercfg-zbzd6\""
Oct 30 21:51:09.545351 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:09.545310    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ea0c6d1-5765-43c8-95c1-19473cc909e2-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-3c4802-7bd9747b5f-sfrmg\" (UID: \"7ea0c6d1-5765-43c8-95c1-19473cc909e2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-3c4802-7bd9747b5f-sfrmg"
Oct 30 21:51:09.545726 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:09.545699    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzm5z\" (UniqueName: \"kubernetes.io/projected/7ea0c6d1-5765-43c8-95c1-19473cc909e2-kube-api-access-fzm5z\") pod \"payload-pre-processing-e2e-worker-w1-3c4802-7bd9747b5f-sfrmg\" (UID: \"7ea0c6d1-5765-43c8-95c1-19473cc909e2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-3c4802-7bd9747b5f-sfrmg"
Oct 30 21:51:09.571820 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:09.571788    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-3c4802-5f6bddd989-bl95r"]
Oct 30 21:51:09.575805 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:09.575779    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-3c4802-5f6bddd989-bl95r"
Oct 30 21:51:09.631826 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:09.631777    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-3c4802-5f6bddd989-bl95r"]
Oct 30 21:51:09.646699 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:09.646664    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c957c3f-753d-4d57-b724-19fdce296860-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-3c4802-5f6bddd989-bl95r\" (UID: \"7c957c3f-753d-4d57-b724-19fdce296860\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-3c4802-5f6bddd989-bl95r"
Oct 30 21:51:09.646869 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:09.646708    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ea0c6d1-5765-43c8-95c1-19473cc909e2-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-3c4802-7bd9747b5f-sfrmg\" (UID: \"7ea0c6d1-5765-43c8-95c1-19473cc909e2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-3c4802-7bd9747b5f-sfrmg"
Oct 30 21:51:09.646869 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:09.646842    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzm5z\" (UniqueName: \"kubernetes.io/projected/7ea0c6d1-5765-43c8-95c1-19473cc909e2-kube-api-access-fzm5z\") pod \"payload-pre-processing-e2e-worker-w1-3c4802-7bd9747b5f-sfrmg\" (UID: \"7ea0c6d1-5765-43c8-95c1-19473cc909e2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-3c4802-7bd9747b5f-sfrmg"
Oct 30 21:51:09.647009 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:09.646933    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wb5sp\" (UniqueName: \"kubernetes.io/projected/7c957c3f-753d-4d57-b724-19fdce296860-kube-api-access-wb5sp\") pod \"payload-processing-e2e-worker-w1-3c4802-5f6bddd989-bl95r\" (UID: \"7c957c3f-753d-4d57-b724-19fdce296860\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-3c4802-5f6bddd989-bl95r"
Oct 30 21:51:09.647499 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:09.647476    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ea0c6d1-5765-43c8-95c1-19473cc909e2-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-3c4802-7bd9747b5f-sfrmg\" (UID: \"7ea0c6d1-5765-43c8-95c1-19473cc909e2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-3c4802-7bd9747b5f-sfrmg"
Oct 30 21:51:09.705505 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:09.705429    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzm5z\" (UniqueName: \"kubernetes.io/projected/7ea0c6d1-5765-43c8-95c1-19473cc909e2-kube-api-access-fzm5z\") pod \"payload-pre-processing-e2e-worker-w1-3c4802-7bd9747b5f-sfrmg\" (UID: \"7ea0c6d1-5765-43c8-95c1-19473cc909e2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-3c4802-7bd9747b5f-sfrmg"
Oct 30 21:51:09.744589 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:09.744558    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-3c4802-7bd9747b5f-sfrmg"
Oct 30 21:51:09.747919 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:09.747639    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wb5sp\" (UniqueName: \"kubernetes.io/projected/7c957c3f-753d-4d57-b724-19fdce296860-kube-api-access-wb5sp\") pod \"payload-processing-e2e-worker-w1-3c4802-5f6bddd989-bl95r\" (UID: \"7c957c3f-753d-4d57-b724-19fdce296860\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-3c4802-5f6bddd989-bl95r"
Oct 30 21:51:09.747919 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:09.747681    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c957c3f-753d-4d57-b724-19fdce296860-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-3c4802-5f6bddd989-bl95r\" (UID: \"7c957c3f-753d-4d57-b724-19fdce296860\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-3c4802-5f6bddd989-bl95r"
Oct 30 21:51:09.748667 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:09.748642    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c957c3f-753d-4d57-b724-19fdce296860-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-3c4802-5f6bddd989-bl95r\" (UID: \"7c957c3f-753d-4d57-b724-19fdce296860\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-3c4802-5f6bddd989-bl95r"
Oct 30 21:51:09.808125 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:09.808092    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wb5sp\" (UniqueName: \"kubernetes.io/projected/7c957c3f-753d-4d57-b724-19fdce296860-kube-api-access-wb5sp\") pod \"payload-processing-e2e-worker-w1-3c4802-5f6bddd989-bl95r\" (UID: \"7c957c3f-753d-4d57-b724-19fdce296860\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-3c4802-5f6bddd989-bl95r"
Oct 30 21:51:09.871770 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:09.871732    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:09.889773 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:09.889737    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-3c4802-5f6bddd989-bl95r"
Oct 30 21:51:10.117449 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:10.117419    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-3c4802-7bd9747b5f-sfrmg"]
Oct 30 21:51:10.118684 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:51:10.118655    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ea0c6d1_5765_43c8_95c1_19473cc909e2.slice/crio-da4997784b87c22f69d56cafa749ffeadae17ed24d7cefd677b0fdec83eccf29 WatchSource:0}: Error finding container da4997784b87c22f69d56cafa749ffeadae17ed24d7cefd677b0fdec83eccf29: Status 404 returned error can't find the container with id da4997784b87c22f69d56cafa749ffeadae17ed24d7cefd677b0fdec83eccf29
Oct 30 21:51:10.274964 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:10.274938    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-3c4802-5f6bddd989-bl95r"]
Oct 30 21:51:10.276964 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:51:10.276933    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c957c3f_753d_4d57_b724_19fdce296860.slice/crio-dc9d0168fb2f679391f8297a048ad0af3f0f02d63ea3d02647083aaf00cd63cb WatchSource:0}: Error finding container dc9d0168fb2f679391f8297a048ad0af3f0f02d63ea3d02647083aaf00cd63cb: Status 404 returned error can't find the container with id dc9d0168fb2f679391f8297a048ad0af3f0f02d63ea3d02647083aaf00cd63cb
Oct 30 21:51:10.422505 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:10.422471    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:10.526822 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:10.526710    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-3c4802-5f6bddd989-bl95r" event={"ID":"7c957c3f-753d-4d57-b724-19fdce296860","Type":"ContainerStarted","Data":"92cf645164171857e2a832f2e90c13f838d580ed26ec8667a285d715ab83c116"}
Oct 30 21:51:10.526822 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:10.526785    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-3c4802-5f6bddd989-bl95r" event={"ID":"7c957c3f-753d-4d57-b724-19fdce296860","Type":"ContainerStarted","Data":"dc9d0168fb2f679391f8297a048ad0af3f0f02d63ea3d02647083aaf00cd63cb"}
Oct 30 21:51:10.526822 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:10.526814    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-3c4802-5f6bddd989-bl95r"
Oct 30 21:51:10.528704 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:10.528677    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-3c4802-7bd9747b5f-sfrmg" event={"ID":"7ea0c6d1-5765-43c8-95c1-19473cc909e2","Type":"ContainerStarted","Data":"99505ec2c53198b22b5f80dd37e60ec33f320237aeaa0fc4ec9208a3794fcd91"}
Oct 30 21:51:10.528857 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:10.528718    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-3c4802-7bd9747b5f-sfrmg" event={"ID":"7ea0c6d1-5765-43c8-95c1-19473cc909e2","Type":"ContainerStarted","Data":"da4997784b87c22f69d56cafa749ffeadae17ed24d7cefd677b0fdec83eccf29"}
Oct 30 21:51:10.528857 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:10.528785    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-3c4802-7bd9747b5f-sfrmg"
Oct 30 21:51:10.602038 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:10.601980    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:10.641648 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:10.641593    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w1-3c4802-5f6bddd989-bl95r" podStartSLOduration=1.641577287 podStartE2EDuration="1.641577287s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:51:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:51:10.638550487 +0000 UTC m=+812.816039106" watchObservedRunningTime="2026-10-30 21:51:10.641577287 +0000 UTC m=+812.819065905"
Oct 30 21:51:10.750892 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:10.750830    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-3c4802-7bd9747b5f-sfrmg" podStartSLOduration=1.750812685 podStartE2EDuration="1.750812685s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:51:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:51:10.747115072 +0000 UTC m=+812.924603690" watchObservedRunningTime="2026-10-30 21:51:10.750812685 +0000 UTC m=+812.928301305"
Oct 30 21:51:10.765289 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:10.764639    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:13.473984 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:13.473952    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:14.480438 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:14.480409    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-4d14e0-8577b8cdc7-tpcg9"
Oct 30 21:51:14.762322 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:14.762221    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47"]
Oct 30 21:51:14.767359 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:14.767333    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47"
Oct 30 21:51:14.818602 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:14.818527    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w3-4d14e0-serving-cert\""
Oct 30 21:51:14.822416 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:14.822382    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47"]
Oct 30 21:51:14.907467 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:14.907429    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f61a7f98-7755-4550-9f9b-fe3cce16f75f-metrics-certs\") pod \"maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47\" (UID: \"f61a7f98-7755-4550-9f9b-fe3cce16f75f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47"
Oct 30 21:51:14.907628 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:14.907475    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9v2d\" (UniqueName: \"kubernetes.io/projected/f61a7f98-7755-4550-9f9b-fe3cce16f75f-kube-api-access-w9v2d\") pod \"maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47\" (UID: \"f61a7f98-7755-4550-9f9b-fe3cce16f75f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47"
Oct 30 21:51:14.907671 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:14.907642    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f61a7f98-7755-4550-9f9b-fe3cce16f75f-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47\" (UID: \"f61a7f98-7755-4550-9f9b-fe3cce16f75f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47"
Oct 30 21:51:14.907778 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:14.907758    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f61a7f98-7755-4550-9f9b-fe3cce16f75f-maas-api-tls\") pod \"maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47\" (UID: \"f61a7f98-7755-4550-9f9b-fe3cce16f75f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47"
Oct 30 21:51:15.009341 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:15.009297    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f61a7f98-7755-4550-9f9b-fe3cce16f75f-metrics-certs\") pod \"maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47\" (UID: \"f61a7f98-7755-4550-9f9b-fe3cce16f75f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47"
Oct 30 21:51:15.009987 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:15.009942    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9v2d\" (UniqueName: \"kubernetes.io/projected/f61a7f98-7755-4550-9f9b-fe3cce16f75f-kube-api-access-w9v2d\") pod \"maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47\" (UID: \"f61a7f98-7755-4550-9f9b-fe3cce16f75f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47"
Oct 30 21:51:15.010106 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:15.010067    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f61a7f98-7755-4550-9f9b-fe3cce16f75f-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47\" (UID: \"f61a7f98-7755-4550-9f9b-fe3cce16f75f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47"
Oct 30 21:51:15.013845 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:15.010171    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f61a7f98-7755-4550-9f9b-fe3cce16f75f-maas-api-tls\") pod \"maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47\" (UID: \"f61a7f98-7755-4550-9f9b-fe3cce16f75f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47"
Oct 30 21:51:15.013845 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:51:15.010480    4871 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w3-4d14e0-serving-cert: secret "maas-api-e2e-worker-w3-4d14e0-serving-cert" not found
Oct 30 21:51:15.013845 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:51:15.010582    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f61a7f98-7755-4550-9f9b-fe3cce16f75f-maas-api-tls podName:f61a7f98-7755-4550-9f9b-fe3cce16f75f nodeName:}" failed. No retries permitted until 2026-10-30 21:51:15.510546523 +0000 UTC m=+817.688035120 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f61a7f98-7755-4550-9f9b-fe3cce16f75f-maas-api-tls") pod "maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47" (UID: "f61a7f98-7755-4550-9f9b-fe3cce16f75f") : secret "maas-api-e2e-worker-w3-4d14e0-serving-cert" not found
Oct 30 21:51:15.015615 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:15.015590    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f61a7f98-7755-4550-9f9b-fe3cce16f75f-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47\" (UID: \"f61a7f98-7755-4550-9f9b-fe3cce16f75f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47"
Oct 30 21:51:15.021696 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:15.020744    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f61a7f98-7755-4550-9f9b-fe3cce16f75f-metrics-certs\") pod \"maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47\" (UID: \"f61a7f98-7755-4550-9f9b-fe3cce16f75f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47"
Oct 30 21:51:15.059555 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:15.059519    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:15.072633 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:15.072602    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9v2d\" (UniqueName: \"kubernetes.io/projected/f61a7f98-7755-4550-9f9b-fe3cce16f75f-kube-api-access-w9v2d\") pod \"maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47\" (UID: \"f61a7f98-7755-4550-9f9b-fe3cce16f75f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47"
Oct 30 21:51:15.218680 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:15.218643    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:15.363818 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:15.363778    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:15.516903 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:15.516866    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f61a7f98-7755-4550-9f9b-fe3cce16f75f-maas-api-tls\") pod \"maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47\" (UID: \"f61a7f98-7755-4550-9f9b-fe3cce16f75f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47"
Oct 30 21:51:15.519368 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:15.519341    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f61a7f98-7755-4550-9f9b-fe3cce16f75f-maas-api-tls\") pod \"maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47\" (UID: \"f61a7f98-7755-4550-9f9b-fe3cce16f75f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47"
Oct 30 21:51:15.681604 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:15.681515    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47"
Oct 30 21:51:16.058907 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:16.058830    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47"]
Oct 30 21:51:16.062257 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:51:16.062204    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf61a7f98_7755_4550_9f9b_fe3cce16f75f.slice/crio-cf4bfe958ddafae63dbc9e8a9ce9b9c0660bf6b2a8b3f993f94c3f400c0c0990 WatchSource:0}: Error finding container cf4bfe958ddafae63dbc9e8a9ce9b9c0660bf6b2a8b3f993f94c3f400c0c0990: Status 404 returned error can't find the container with id cf4bfe958ddafae63dbc9e8a9ce9b9c0660bf6b2a8b3f993f94c3f400c0c0990
Oct 30 21:51:16.570266 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:16.570214    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47" event={"ID":"f61a7f98-7755-4550-9f9b-fe3cce16f75f","Type":"ContainerStarted","Data":"cf4bfe958ddafae63dbc9e8a9ce9b9c0660bf6b2a8b3f993f94c3f400c0c0990"}
Oct 30 21:51:17.582262 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:17.581947    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47" event={"ID":"f61a7f98-7755-4550-9f9b-fe3cce16f75f","Type":"ContainerStarted","Data":"3c07939c15520f06116434e4c504c8154745c454f6fcc19290fa375df072be08"}
Oct 30 21:51:17.582730 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:17.582422    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47"
Oct 30 21:51:17.693601 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:17.693546    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47" podStartSLOduration=2.776075331 podStartE2EDuration="3.693532806s" totalImagesPullingTime="917.457475ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:51:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:51:17.693413527 +0000 UTC m=+819.870902144" watchObservedRunningTime="2026-10-30 21:51:17.693532806 +0000 UTC m=+819.871021424"
Oct 30 21:51:18.819121 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:18.819087    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:20.098375 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:20.098341    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:21.538044 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:21.538013    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-3c4802-5f6bddd989-bl95r"
Oct 30 21:51:21.538540 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:21.538087    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-3c4802-7bd9747b5f-sfrmg"
Oct 30 21:51:23.593902 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:23.593873    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47"
Oct 30 21:51:24.001971 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:24.001933    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:24.561849 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:24.561807    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:24.717012 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:24.716982    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:24.871572 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:24.871540    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:29.459582 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:29.459546    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:29.627976 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:29.627939    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:29.795944 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:29.795861    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:32.430090 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:32.430056    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:34.302471 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:34.302433    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:35.569084 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:51:35.568990    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:51:35.719916 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:35.719816    4871 generic.go:356] "Generic (PLEG): container finished" podID="1969a36c-2ac9-4972-8a9e-e84e39682a28" containerID="cb13b4186f0a942b24f7f268d5138b480ab6df06de9ac90c33879029c3618849" exitCode=0
Oct 30 21:51:35.719916 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:35.719892    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx" event={"ID":"1969a36c-2ac9-4972-8a9e-e84e39682a28","Type":"ContainerDied","Data":"cb13b4186f0a942b24f7f268d5138b480ab6df06de9ac90c33879029c3618849"}
Oct 30 21:51:35.720369 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:35.720351    4871 scope.go:122] "RemoveContainer" containerID="cb13b4186f0a942b24f7f268d5138b480ab6df06de9ac90c33879029c3618849"
Oct 30 21:51:35.722436 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:35.722396    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:51:35.722552 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:35.722496    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:51:35.722552 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:35.722541    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:51:36.749260 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:36.748613    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx" event={"ID":"1969a36c-2ac9-4972-8a9e-e84e39682a28","Type":"ContainerStarted","Data":"d22346676e75efcd76c9470ac6e977700c4c39d07cdcffc3438a067bf23a5dae"}
Oct 30 21:51:37.616260 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:37.610633    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"
Oct 30 21:51:37.618328 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:37.618294    4871 patch_prober.go:29] interesting pod/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 21:51:37.618568 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:37.618538    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx" podUID="1969a36c-2ac9-4972-8a9e-e84e39682a28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 21:51:38.622329 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:38.613722    4871 patch_prober.go:29] interesting pod/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 21:51:38.622329 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:38.613784    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx" podUID="1969a36c-2ac9-4972-8a9e-e84e39682a28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 21:51:39.222262 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:39.222183    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:39.401567 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:39.401527    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"
Oct 30 21:51:39.610798 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:39.610700    4871 patch_prober.go:29] interesting pod/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 21:51:39.611104 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:39.611073    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx" podUID="1969a36c-2ac9-4972-8a9e-e84e39682a28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 21:51:39.787448 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:39.787415    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:39.955687 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:39.955071    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:40.537274 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:40.533766    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:40.618763 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:40.618710    4871 patch_prober.go:29] interesting pod/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 21:51:40.619082 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:40.619050    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx" podUID="1969a36c-2ac9-4972-8a9e-e84e39682a28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 21:51:40.690231 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:40.690196    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:40.848199 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:40.848108    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:41.622264 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:41.607702    4871 patch_prober.go:29] interesting pod/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 21:51:41.622264 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:41.607767    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx" podUID="1969a36c-2ac9-4972-8a9e-e84e39682a28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 21:51:42.612447 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:42.612411    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"
Oct 30 21:51:42.613634 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:42.613610    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"
Oct 30 21:51:44.013196 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:44.013153    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:45.828997 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:45.828960    4871 generic.go:356] "Generic (PLEG): container finished" podID="f3798606-a072-44a5-891c-4588de3c6ff1" containerID="32f8069ac79b7a359568aa080a026b48d32973596ce134cef15097cb136fcddf" exitCode=0
Oct 30 21:51:45.829571 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:45.829036    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm" event={"ID":"f3798606-a072-44a5-891c-4588de3c6ff1","Type":"ContainerDied","Data":"32f8069ac79b7a359568aa080a026b48d32973596ce134cef15097cb136fcddf"}
Oct 30 21:51:45.829571 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:45.829450    4871 scope.go:122] "RemoveContainer" containerID="32f8069ac79b7a359568aa080a026b48d32973596ce134cef15097cb136fcddf"
Oct 30 21:51:45.830751 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:45.830727    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:51:45.830809 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:45.830791    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:51:45.830847 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:45.830828    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:51:46.842905 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:46.842870    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm" event={"ID":"f3798606-a072-44a5-891c-4588de3c6ff1","Type":"ContainerStarted","Data":"e89e1ca44b46da72cb1105b3c1750a576e4432ad389c7f2122844b9f9c5568dc"}
Oct 30 21:51:47.696364 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:47.696325    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"
Oct 30 21:51:47.696555 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:47.696503    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 21:51:47.696603 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:47.696555    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm" podUID="f3798606-a072-44a5-891c-4588de3c6ff1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 21:51:48.697223 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:48.697184    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 21:51:48.697679 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:48.697279    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm" podUID="f3798606-a072-44a5-891c-4588de3c6ff1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 21:51:48.849211 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:48.849174    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:49.696407 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:49.696368    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 21:51:49.696591 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:49.696428    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm" podUID="f3798606-a072-44a5-891c-4588de3c6ff1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 21:51:50.696842 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:50.696802    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 21:51:50.697428 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:50.696863    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm" podUID="f3798606-a072-44a5-891c-4588de3c6ff1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 21:51:51.159006 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:51.158970    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:51.696659 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:51.696615    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 21:51:51.696870 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:51.696688    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm" podUID="f3798606-a072-44a5-891c-4588de3c6ff1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 21:51:52.701020 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:52.700984    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"
Oct 30 21:51:52.701540 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:52.701285    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"
Oct 30 21:51:52.702075 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:52.702051    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"
Oct 30 21:51:53.717504 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:53.717465    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:58.311636 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:51:58.311582    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:01.834017 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:52:01.833982    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:01.990014 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:52:01.989980    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:03.265421 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:52:03.265382    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:03.434201 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:52:03.434166    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:03.813527 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:52:03.813478    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:04.074397 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:52:04.074316    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:04.253519 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:52:04.253448    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:05.567926 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:52:05.567881    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:52:07.901876 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:52:07.901836    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:12.442058 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:52:12.442022    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:13.066635 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:52:13.066602    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:17.101633 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:52:17.101576    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:21.746619 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:52:21.746575    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:23.708368 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:52:23.708328    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:26.290616 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:52:26.290582    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:30.820189 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:52:30.820155    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:32.773489 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:52:32.773451    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:35.420798 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:52:35.418156    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:35.570341 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:52:35.570302    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:52:38.079920 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:52:38.079886    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx_8d401654-8a4c-4a79-b2d3-612765312421/extract/0.log"
Oct 30 21:52:38.081800 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:52:38.081772    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx_8d401654-8a4c-4a79-b2d3-612765312421/extract/0.log"
Oct 30 21:52:38.092512 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:52:38.092485    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/2.log"
Oct 30 21:52:38.094212 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:52:38.094191    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/2.log"
Oct 30 21:52:38.099793 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:52:38.099773    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzntd_2360fddc-52a5-42a5-82ca-a05aff84d217/ovn-acl-logging/0.log"
Oct 30 21:52:38.101343 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:52:38.101323    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzntd_2360fddc-52a5-42a5-82ca-a05aff84d217/ovn-acl-logging/0.log"
Oct 30 21:52:39.072602 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:52:39.072564    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:39.967293 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:52:39.966650    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:43.226133 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:52:43.226093    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:43.865936 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:52:43.865885    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:44.134044 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:52:44.133955    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:44.510813 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:52:44.510730    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:49.161631 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:52:49.161544    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:53.706127 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:52:53.706093    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:58.201950 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:52:58.201907    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:59.540538 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:52:59.540499    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:02.860556 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:02.858767    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:05.569205 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:53:05.569161    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:53:07.411893 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:07.411846    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:12.110989 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:12.110951    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:14.982551 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:14.982423    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"]
Oct 30 21:53:14.983178 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:14.982839    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v" podUID="2c5a0598-58b9-4032-8e23-27daf0444149" containerName="istio-proxy" containerID="cri-o://b3f280e29014840e3e939e906b261a2640c278f1f407db4b98ddffd805301b73" gracePeriod=30
Oct 30 21:53:16.709603 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:16.709555    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:20.242971 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.242939    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"
Oct 30 21:53:20.350325 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.350202    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2c5a0598-58b9-4032-8e23-27daf0444149-istio-token\" (UniqueName: \"kubernetes.io/projected/2c5a0598-58b9-4032-8e23-27daf0444149-istio-token\") pod \"2c5a0598-58b9-4032-8e23-27daf0444149\" (UID: \"2c5a0598-58b9-4032-8e23-27daf0444149\") "
Oct 30 21:53:20.350325 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.350300    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c5a0598-58b9-4032-8e23-27daf0444149-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c5a0598-58b9-4032-8e23-27daf0444149-istio-data\") pod \"2c5a0598-58b9-4032-8e23-27daf0444149\" (UID: \"2c5a0598-58b9-4032-8e23-27daf0444149\") "
Oct 30 21:53:20.350565 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.350333    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c5a0598-58b9-4032-8e23-27daf0444149-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c5a0598-58b9-4032-8e23-27daf0444149-credential-socket\") pod \"2c5a0598-58b9-4032-8e23-27daf0444149\" (UID: \"2c5a0598-58b9-4032-8e23-27daf0444149\") "
Oct 30 21:53:20.350565 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.350366    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c5a0598-58b9-4032-8e23-27daf0444149-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c5a0598-58b9-4032-8e23-27daf0444149-istio-envoy\") pod \"2c5a0598-58b9-4032-8e23-27daf0444149\" (UID: \"2c5a0598-58b9-4032-8e23-27daf0444149\") "
Oct 30 21:53:20.350565 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.350397    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2c5a0598-58b9-4032-8e23-27daf0444149-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c5a0598-58b9-4032-8e23-27daf0444149-istio-podinfo\") pod \"2c5a0598-58b9-4032-8e23-27daf0444149\" (UID: \"2c5a0598-58b9-4032-8e23-27daf0444149\") "
Oct 30 21:53:20.350565 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.350507    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2c5a0598-58b9-4032-8e23-27daf0444149-kube-api-access-nn9gc\" (UniqueName: \"kubernetes.io/projected/2c5a0598-58b9-4032-8e23-27daf0444149-kube-api-access-nn9gc\") pod \"2c5a0598-58b9-4032-8e23-27daf0444149\" (UID: \"2c5a0598-58b9-4032-8e23-27daf0444149\") "
Oct 30 21:53:20.350948 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.350564    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c5a0598-58b9-4032-8e23-27daf0444149-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c5a0598-58b9-4032-8e23-27daf0444149-workload-socket\") pod \"2c5a0598-58b9-4032-8e23-27daf0444149\" (UID: \"2c5a0598-58b9-4032-8e23-27daf0444149\") "
Oct 30 21:53:20.351082 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.350984    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2c5a0598-58b9-4032-8e23-27daf0444149-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c5a0598-58b9-4032-8e23-27daf0444149-istiod-ca-cert\") pod \"2c5a0598-58b9-4032-8e23-27daf0444149\" (UID: \"2c5a0598-58b9-4032-8e23-27daf0444149\") "
Oct 30 21:53:20.351082 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.350623    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c5a0598-58b9-4032-8e23-27daf0444149-istio-data" pod "2c5a0598-58b9-4032-8e23-27daf0444149" (UID: "2c5a0598-58b9-4032-8e23-27daf0444149"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:53:20.351082 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.351026    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c5a0598-58b9-4032-8e23-27daf0444149-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c5a0598-58b9-4032-8e23-27daf0444149-workload-certs\") pod \"2c5a0598-58b9-4032-8e23-27daf0444149\" (UID: \"2c5a0598-58b9-4032-8e23-27daf0444149\") "
Oct 30 21:53:20.351082 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.350665    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c5a0598-58b9-4032-8e23-27daf0444149-credential-socket" pod "2c5a0598-58b9-4032-8e23-27daf0444149" (UID: "2c5a0598-58b9-4032-8e23-27daf0444149"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:53:20.351350 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.350803    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c5a0598-58b9-4032-8e23-27daf0444149-workload-socket" pod "2c5a0598-58b9-4032-8e23-27daf0444149" (UID: "2c5a0598-58b9-4032-8e23-27daf0444149"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:53:20.351471 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.351443    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c5a0598-58b9-4032-8e23-27daf0444149-workload-certs" pod "2c5a0598-58b9-4032-8e23-27daf0444149" (UID: "2c5a0598-58b9-4032-8e23-27daf0444149"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:53:20.351566 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.351450    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c5a0598-58b9-4032-8e23-27daf0444149-istiod-ca-cert" pod "2c5a0598-58b9-4032-8e23-27daf0444149" (UID: "2c5a0598-58b9-4032-8e23-27daf0444149"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:53:20.351625 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.351598    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c5a0598-58b9-4032-8e23-27daf0444149-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:20.351625 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.351619    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c5a0598-58b9-4032-8e23-27daf0444149-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:20.351738 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.351634    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c5a0598-58b9-4032-8e23-27daf0444149-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:20.351738 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.351648    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c5a0598-58b9-4032-8e23-27daf0444149-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:20.351738 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.351661    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c5a0598-58b9-4032-8e23-27daf0444149-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:20.352863 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.352836    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c5a0598-58b9-4032-8e23-27daf0444149-istio-token" pod "2c5a0598-58b9-4032-8e23-27daf0444149" (UID: "2c5a0598-58b9-4032-8e23-27daf0444149"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:53:20.353049 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.353019    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c5a0598-58b9-4032-8e23-27daf0444149-istio-envoy" pod "2c5a0598-58b9-4032-8e23-27daf0444149" (UID: "2c5a0598-58b9-4032-8e23-27daf0444149"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:53:20.353121 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.353045    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2c5a0598-58b9-4032-8e23-27daf0444149-istio-podinfo" pod "2c5a0598-58b9-4032-8e23-27daf0444149" (UID: "2c5a0598-58b9-4032-8e23-27daf0444149"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:53:20.353185 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.353164    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c5a0598-58b9-4032-8e23-27daf0444149-kube-api-access-nn9gc" pod "2c5a0598-58b9-4032-8e23-27daf0444149" (UID: "2c5a0598-58b9-4032-8e23-27daf0444149"). InnerVolumeSpecName "kube-api-access-nn9gc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:53:20.452787 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.452737    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c5a0598-58b9-4032-8e23-27daf0444149-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:20.452787 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.452786    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c5a0598-58b9-4032-8e23-27daf0444149-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:20.453004 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.452802    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c5a0598-58b9-4032-8e23-27daf0444149-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:20.453004 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.452818    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nn9gc\" (UniqueName: \"kubernetes.io/projected/2c5a0598-58b9-4032-8e23-27daf0444149-kube-api-access-nn9gc\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:20.483973 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.483937    4871 generic.go:356] "Generic (PLEG): container finished" podID="2c5a0598-58b9-4032-8e23-27daf0444149" containerID="b3f280e29014840e3e939e906b261a2640c278f1f407db4b98ddffd805301b73" exitCode=0
Oct 30 21:53:20.484228 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.483995    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"
Oct 30 21:53:20.484228 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.484028    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v" event={"ID":"2c5a0598-58b9-4032-8e23-27daf0444149","Type":"ContainerDied","Data":"b3f280e29014840e3e939e906b261a2640c278f1f407db4b98ddffd805301b73"}
Oct 30 21:53:20.484228 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.484084    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v" event={"ID":"2c5a0598-58b9-4032-8e23-27daf0444149","Type":"ContainerDied","Data":"ad56eba65ee54bb74ff31199b56b2486516b6d72e8055137eba428bffb07b8da"}
Oct 30 21:53:20.484228 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.484110    4871 scope.go:122] "RemoveContainer" containerID="b3f280e29014840e3e939e906b261a2640c278f1f407db4b98ddffd805301b73"
Oct 30 21:53:20.495563 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.495540    4871 scope.go:122] "RemoveContainer" containerID="4c2b14109615eae580160381e381270bccd8598b240e30f91c9ee2f4c7961fa3"
Oct 30 21:53:20.505340 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.505320    4871 scope.go:122] "RemoveContainer" containerID="b3f280e29014840e3e939e906b261a2640c278f1f407db4b98ddffd805301b73"
Oct 30 21:53:20.505651 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:53:20.505632    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3f280e29014840e3e939e906b261a2640c278f1f407db4b98ddffd805301b73\": container with ID starting with b3f280e29014840e3e939e906b261a2640c278f1f407db4b98ddffd805301b73 not found: ID does not exist" containerID="b3f280e29014840e3e939e906b261a2640c278f1f407db4b98ddffd805301b73"
Oct 30 21:53:20.505717 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.505661    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3f280e29014840e3e939e906b261a2640c278f1f407db4b98ddffd805301b73"} err="failed to get container status \"b3f280e29014840e3e939e906b261a2640c278f1f407db4b98ddffd805301b73\": rpc error: code = NotFound desc = could not find container \"b3f280e29014840e3e939e906b261a2640c278f1f407db4b98ddffd805301b73\": container with ID starting with b3f280e29014840e3e939e906b261a2640c278f1f407db4b98ddffd805301b73 not found: ID does not exist"
Oct 30 21:53:20.505717 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.505679    4871 scope.go:122] "RemoveContainer" containerID="4c2b14109615eae580160381e381270bccd8598b240e30f91c9ee2f4c7961fa3"
Oct 30 21:53:20.505914 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:53:20.505896    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c2b14109615eae580160381e381270bccd8598b240e30f91c9ee2f4c7961fa3\": container with ID starting with 4c2b14109615eae580160381e381270bccd8598b240e30f91c9ee2f4c7961fa3 not found: ID does not exist" containerID="4c2b14109615eae580160381e381270bccd8598b240e30f91c9ee2f4c7961fa3"
Oct 30 21:53:20.505975 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.505921    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c2b14109615eae580160381e381270bccd8598b240e30f91c9ee2f4c7961fa3"} err="failed to get container status \"4c2b14109615eae580160381e381270bccd8598b240e30f91c9ee2f4c7961fa3\": rpc error: code = NotFound desc = could not find container \"4c2b14109615eae580160381e381270bccd8598b240e30f91c9ee2f4c7961fa3\": container with ID starting with 4c2b14109615eae580160381e381270bccd8598b240e30f91c9ee2f4c7961fa3 not found: ID does not exist"
Oct 30 21:53:20.633669 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.633624    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:20.641829 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.641798    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"]
Oct 30 21:53:20.645557 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.645525    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-hkg2v"]
Oct 30 21:53:20.787332 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:20.787301    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:21.297330 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:21.297287    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:21.959992 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:21.959961    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2c5a0598-58b9-4032-8e23-27daf0444149" path="/var/lib/kubelet/pods/2c5a0598-58b9-4032-8e23-27daf0444149/volumes"
Oct 30 21:53:25.959088 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:25.959051    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:28.855976 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:28.855937    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:30.543985 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:30.543933    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:33.134522 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:33.134481    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:33.296473 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:33.296430    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:35.139832 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:35.139790    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:35.569743 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:53:35.569639    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:53:39.759407 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:39.759369    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:44.350852 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:44.350805    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:48.969715 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:48.969675    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:53.689251 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:53.689209    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:55.492753 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:55.492718    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8"]
Oct 30 21:53:55.493270 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:55.493230    4871 cpu_manager.go:414] "containerMap: removing container" podUID="2c5a0598-58b9-4032-8e23-27daf0444149" containerName="istio-proxy"
Oct 30 21:53:55.493323 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:55.493273    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2c5a0598-58b9-4032-8e23-27daf0444149" containerName="istio-proxy"
Oct 30 21:53:55.493323 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:55.493304    4871 cpu_manager.go:414] "containerMap: removing container" podUID="2c5a0598-58b9-4032-8e23-27daf0444149" containerName="istio-proxy"
Oct 30 21:53:55.493323 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:55.493312    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2c5a0598-58b9-4032-8e23-27daf0444149" containerName="istio-proxy"
Oct 30 21:53:55.493470 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:55.493458    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="2c5a0598-58b9-4032-8e23-27daf0444149" containerName="istio-proxy"
Oct 30 21:53:55.496764 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:55.496742    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8"
Oct 30 21:53:55.594182 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:55.594145    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-3c4802\"/\"premium-w1-3c4802-kserve-self-signed-certs\""
Oct 30 21:53:55.594182 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:55.594147    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-3c4802\"/\"kube-root-ca.crt\""
Oct 30 21:53:55.594182 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:55.594192    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-3c4802\"/\"openshift-service-ca.crt\""
Oct 30 21:53:55.594618 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:55.594595    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-3c4802\"/\"default-dockercfg-7bt4g\""
Oct 30 21:53:55.598557 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:55.598537    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8"]
Oct 30 21:53:55.601929 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:55.601907    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84329f90-dc35-41dc-b593-12c556e80f73-home\") pod \"premium-w1-3c4802-kserve-7c5849d557-fs6r8\" (UID: \"84329f90-dc35-41dc-b593-12c556e80f73\") " pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8"
Oct 30 21:53:55.602048 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:55.601943    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84329f90-dc35-41dc-b593-12c556e80f73-tmp-dir\") pod \"premium-w1-3c4802-kserve-7c5849d557-fs6r8\" (UID: \"84329f90-dc35-41dc-b593-12c556e80f73\") " pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8"
Oct 30 21:53:55.602048 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:55.601959    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84329f90-dc35-41dc-b593-12c556e80f73-tls-certs\") pod \"premium-w1-3c4802-kserve-7c5849d557-fs6r8\" (UID: \"84329f90-dc35-41dc-b593-12c556e80f73\") " pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8"
Oct 30 21:53:55.602139 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:55.602123    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84329f90-dc35-41dc-b593-12c556e80f73-model-cache\") pod \"premium-w1-3c4802-kserve-7c5849d557-fs6r8\" (UID: \"84329f90-dc35-41dc-b593-12c556e80f73\") " pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8"
Oct 30 21:53:55.602178 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:55.602156    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vxg7\" (UniqueName: \"kubernetes.io/projected/84329f90-dc35-41dc-b593-12c556e80f73-kube-api-access-2vxg7\") pod \"premium-w1-3c4802-kserve-7c5849d557-fs6r8\" (UID: \"84329f90-dc35-41dc-b593-12c556e80f73\") " pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8"
Oct 30 21:53:55.602225 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:55.602185    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84329f90-dc35-41dc-b593-12c556e80f73-dshm\") pod \"premium-w1-3c4802-kserve-7c5849d557-fs6r8\" (UID: \"84329f90-dc35-41dc-b593-12c556e80f73\") " pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8"
Oct 30 21:53:55.621387 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:55.621335    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:55.702965 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:55.702929    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84329f90-dc35-41dc-b593-12c556e80f73-tmp-dir\") pod \"premium-w1-3c4802-kserve-7c5849d557-fs6r8\" (UID: \"84329f90-dc35-41dc-b593-12c556e80f73\") " pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8"
Oct 30 21:53:55.702965 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:55.702970    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84329f90-dc35-41dc-b593-12c556e80f73-tls-certs\") pod \"premium-w1-3c4802-kserve-7c5849d557-fs6r8\" (UID: \"84329f90-dc35-41dc-b593-12c556e80f73\") " pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8"
Oct 30 21:53:55.703209 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:55.703042    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84329f90-dc35-41dc-b593-12c556e80f73-model-cache\") pod \"premium-w1-3c4802-kserve-7c5849d557-fs6r8\" (UID: \"84329f90-dc35-41dc-b593-12c556e80f73\") " pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8"
Oct 30 21:53:55.703209 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:55.703172    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vxg7\" (UniqueName: \"kubernetes.io/projected/84329f90-dc35-41dc-b593-12c556e80f73-kube-api-access-2vxg7\") pod \"premium-w1-3c4802-kserve-7c5849d557-fs6r8\" (UID: \"84329f90-dc35-41dc-b593-12c556e80f73\") " pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8"
Oct 30 21:53:55.703316 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:55.703218    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84329f90-dc35-41dc-b593-12c556e80f73-dshm\") pod \"premium-w1-3c4802-kserve-7c5849d557-fs6r8\" (UID: \"84329f90-dc35-41dc-b593-12c556e80f73\") " pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8"
Oct 30 21:53:55.703316 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:55.703297    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84329f90-dc35-41dc-b593-12c556e80f73-home\") pod \"premium-w1-3c4802-kserve-7c5849d557-fs6r8\" (UID: \"84329f90-dc35-41dc-b593-12c556e80f73\") " pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8"
Oct 30 21:53:55.703665 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:55.703399    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84329f90-dc35-41dc-b593-12c556e80f73-tmp-dir\") pod \"premium-w1-3c4802-kserve-7c5849d557-fs6r8\" (UID: \"84329f90-dc35-41dc-b593-12c556e80f73\") " pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8"
Oct 30 21:53:55.703665 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:55.703475    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84329f90-dc35-41dc-b593-12c556e80f73-model-cache\") pod \"premium-w1-3c4802-kserve-7c5849d557-fs6r8\" (UID: \"84329f90-dc35-41dc-b593-12c556e80f73\") " pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8"
Oct 30 21:53:55.703665 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:55.703606    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84329f90-dc35-41dc-b593-12c556e80f73-home\") pod \"premium-w1-3c4802-kserve-7c5849d557-fs6r8\" (UID: \"84329f90-dc35-41dc-b593-12c556e80f73\") " pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8"
Oct 30 21:53:55.705527 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:55.705492    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84329f90-dc35-41dc-b593-12c556e80f73-dshm\") pod \"premium-w1-3c4802-kserve-7c5849d557-fs6r8\" (UID: \"84329f90-dc35-41dc-b593-12c556e80f73\") " pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8"
Oct 30 21:53:55.705691 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:55.705673    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84329f90-dc35-41dc-b593-12c556e80f73-tls-certs\") pod \"premium-w1-3c4802-kserve-7c5849d557-fs6r8\" (UID: \"84329f90-dc35-41dc-b593-12c556e80f73\") " pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8"
Oct 30 21:53:55.762070 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:55.761978    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vxg7\" (UniqueName: \"kubernetes.io/projected/84329f90-dc35-41dc-b593-12c556e80f73-kube-api-access-2vxg7\") pod \"premium-w1-3c4802-kserve-7c5849d557-fs6r8\" (UID: \"84329f90-dc35-41dc-b593-12c556e80f73\") " pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8"
Oct 30 21:53:55.821643 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:55.821609    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8"
Oct 30 21:53:56.214138 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:56.214111    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8"]
Oct 30 21:53:56.754856 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:56.754817    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8" event={"ID":"84329f90-dc35-41dc-b593-12c556e80f73","Type":"ContainerStarted","Data":"a05af2779fb51d291e15834dbe63876fa4c5384c1e340a620970f057943f1a4b"}
Oct 30 21:53:58.456550 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:58.456515    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:58.778441 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:58.778343    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8" event={"ID":"84329f90-dc35-41dc-b593-12c556e80f73","Type":"ContainerStarted","Data":"ca1e9c929acaca04d466dfd6fbcc43cf1562705a2b1366924bdc4e16027eda85"}
Oct 30 21:53:58.822028 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:58.821973    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8"
Oct 30 21:53:58.894661 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:53:58.894602    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8" podStartSLOduration=2.167844034 podStartE2EDuration="3.894584324s" totalImagesPullingTime="1.72674029s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:53:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:53:58.891705532 +0000 UTC m=+981.069194145" watchObservedRunningTime="2026-10-30 21:53:58.894584324 +0000 UTC m=+981.072072940"
Oct 30 21:54:03.246548 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:03.246512    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:04.478467 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:04.478435    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:04.651799 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:04.651756    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:05.569977 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:54:05.569941    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:54:05.822151 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:05.822050    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8"
Oct 30 21:54:05.835569 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:05.835541    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8"
Oct 30 21:54:05.847158 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:05.847128    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8"
Oct 30 21:54:06.782734 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:06.782690    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"]
Oct 30 21:54:06.783401 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:06.783375    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="2c5a0598-58b9-4032-8e23-27daf0444149" containerName="istio-proxy"
Oct 30 21:54:06.787076 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:06.787056    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"
Oct 30 21:54:06.836564 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:06.836529    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-56f3fe-openshift-default-dockercfg-48wbx\""
Oct 30 21:54:06.887886 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:06.887852    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"]
Oct 30 21:54:06.936835 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:06.936794    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9a09763-b456-469d-80c7-a787456003f4-istio-token\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj\" (UID: \"d9a09763-b456-469d-80c7-a787456003f4\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"
Oct 30 21:54:06.937022 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:06.936863    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9a09763-b456-469d-80c7-a787456003f4-workload-certs\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj\" (UID: \"d9a09763-b456-469d-80c7-a787456003f4\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"
Oct 30 21:54:06.937022 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:06.936930    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9a09763-b456-469d-80c7-a787456003f4-istio-podinfo\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj\" (UID: \"d9a09763-b456-469d-80c7-a787456003f4\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"
Oct 30 21:54:06.937524 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:06.937022    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a09763-b456-469d-80c7-a787456003f4-credential-socket\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj\" (UID: \"d9a09763-b456-469d-80c7-a787456003f4\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"
Oct 30 21:54:06.937524 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:06.937081    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6rsf\" (UniqueName: \"kubernetes.io/projected/d9a09763-b456-469d-80c7-a787456003f4-kube-api-access-t6rsf\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj\" (UID: \"d9a09763-b456-469d-80c7-a787456003f4\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"
Oct 30 21:54:06.937524 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:06.937302    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9a09763-b456-469d-80c7-a787456003f4-istiod-ca-cert\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj\" (UID: \"d9a09763-b456-469d-80c7-a787456003f4\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"
Oct 30 21:54:06.937524 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:06.937478    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9a09763-b456-469d-80c7-a787456003f4-istio-data\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj\" (UID: \"d9a09763-b456-469d-80c7-a787456003f4\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"
Oct 30 21:54:06.937709 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:06.937571    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9a09763-b456-469d-80c7-a787456003f4-istio-envoy\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj\" (UID: \"d9a09763-b456-469d-80c7-a787456003f4\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"
Oct 30 21:54:06.937709 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:06.937607    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a09763-b456-469d-80c7-a787456003f4-workload-socket\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj\" (UID: \"d9a09763-b456-469d-80c7-a787456003f4\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"
Oct 30 21:54:07.038262 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:07.038152    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9a09763-b456-469d-80c7-a787456003f4-istio-data\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj\" (UID: \"d9a09763-b456-469d-80c7-a787456003f4\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"
Oct 30 21:54:07.038262 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:07.038200    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9a09763-b456-469d-80c7-a787456003f4-istio-envoy\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj\" (UID: \"d9a09763-b456-469d-80c7-a787456003f4\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"
Oct 30 21:54:07.038262 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:07.038219    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a09763-b456-469d-80c7-a787456003f4-workload-socket\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj\" (UID: \"d9a09763-b456-469d-80c7-a787456003f4\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"
Oct 30 21:54:07.038262 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:07.038255    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9a09763-b456-469d-80c7-a787456003f4-istio-token\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj\" (UID: \"d9a09763-b456-469d-80c7-a787456003f4\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"
Oct 30 21:54:07.038621 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:07.038304    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9a09763-b456-469d-80c7-a787456003f4-workload-certs\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj\" (UID: \"d9a09763-b456-469d-80c7-a787456003f4\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"
Oct 30 21:54:07.038621 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:07.038567    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9a09763-b456-469d-80c7-a787456003f4-istio-podinfo\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj\" (UID: \"d9a09763-b456-469d-80c7-a787456003f4\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"
Oct 30 21:54:07.038719 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:07.038640    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a09763-b456-469d-80c7-a787456003f4-credential-socket\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj\" (UID: \"d9a09763-b456-469d-80c7-a787456003f4\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"
Oct 30 21:54:07.038719 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:07.038652    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9a09763-b456-469d-80c7-a787456003f4-istio-data\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj\" (UID: \"d9a09763-b456-469d-80c7-a787456003f4\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"
Oct 30 21:54:07.038719 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:07.038698    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6rsf\" (UniqueName: \"kubernetes.io/projected/d9a09763-b456-469d-80c7-a787456003f4-kube-api-access-t6rsf\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj\" (UID: \"d9a09763-b456-469d-80c7-a787456003f4\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"
Oct 30 21:54:07.038844 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:07.038719    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a09763-b456-469d-80c7-a787456003f4-workload-socket\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj\" (UID: \"d9a09763-b456-469d-80c7-a787456003f4\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"
Oct 30 21:54:07.038844 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:07.038730    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9a09763-b456-469d-80c7-a787456003f4-istiod-ca-cert\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj\" (UID: \"d9a09763-b456-469d-80c7-a787456003f4\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"
Oct 30 21:54:07.038844 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:07.038791    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9a09763-b456-469d-80c7-a787456003f4-workload-certs\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj\" (UID: \"d9a09763-b456-469d-80c7-a787456003f4\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"
Oct 30 21:54:07.038953 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:07.038931    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a09763-b456-469d-80c7-a787456003f4-credential-socket\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj\" (UID: \"d9a09763-b456-469d-80c7-a787456003f4\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"
Oct 30 21:54:07.039284 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:07.039258    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9a09763-b456-469d-80c7-a787456003f4-istiod-ca-cert\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj\" (UID: \"d9a09763-b456-469d-80c7-a787456003f4\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"
Oct 30 21:54:07.040923 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:07.040893    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9a09763-b456-469d-80c7-a787456003f4-istio-envoy\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj\" (UID: \"d9a09763-b456-469d-80c7-a787456003f4\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"
Oct 30 21:54:07.041076 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:07.041056    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9a09763-b456-469d-80c7-a787456003f4-istio-podinfo\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj\" (UID: \"d9a09763-b456-469d-80c7-a787456003f4\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"
Oct 30 21:54:07.097541 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:07.097506    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6rsf\" (UniqueName: \"kubernetes.io/projected/d9a09763-b456-469d-80c7-a787456003f4-kube-api-access-t6rsf\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj\" (UID: \"d9a09763-b456-469d-80c7-a787456003f4\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"
Oct 30 21:54:07.103021 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:07.102988    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9a09763-b456-469d-80c7-a787456003f4-istio-token\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj\" (UID: \"d9a09763-b456-469d-80c7-a787456003f4\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"
Oct 30 21:54:07.401291 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:07.401260    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"
Oct 30 21:54:07.830517 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:07.830485    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"]
Oct 30 21:54:07.831192 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:54:07.831156    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9a09763_b456_469d_80c7_a787456003f4.slice/crio-7540e858f52ed3575b6e1a9aca9788822210b98b1040f4a57f4c3603c147bd37 WatchSource:0}: Error finding container 7540e858f52ed3575b6e1a9aca9788822210b98b1040f4a57f4c3603c147bd37: Status 404 returned error can't find the container with id 7540e858f52ed3575b6e1a9aca9788822210b98b1040f4a57f4c3603c147bd37
Oct 30 21:54:07.834691 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:07.834655    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:54:07.834786 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:07.834723    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:54:07.834786 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:07.834751    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:54:07.848581 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:07.848556    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj" event={"ID":"d9a09763-b456-469d-80c7-a787456003f4","Type":"ContainerStarted","Data":"7540e858f52ed3575b6e1a9aca9788822210b98b1040f4a57f4c3603c147bd37"}
Oct 30 21:54:07.919412 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:07.919380    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:08.857548 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:08.857493    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj" event={"ID":"d9a09763-b456-469d-80c7-a787456003f4","Type":"ContainerStarted","Data":"683650a931b7ff26ab034048cd5f25a91bb183c0453af8191f7c446ae45fc24c"}
Oct 30 21:54:08.985671 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:08.985603    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj" podStartSLOduration=2.985584722 podStartE2EDuration="2.985584722s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:54:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:08.98178682 +0000 UTC m=+991.159275437" watchObservedRunningTime="2026-10-30 21:54:08.985584722 +0000 UTC m=+991.163073346"
Oct 30 21:54:09.409262 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:09.402304    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"
Oct 30 21:54:09.427415 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:09.427369    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Oct 30 21:54:09.427650 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:09.427432    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj" podUID="d9a09763-b456-469d-80c7-a787456003f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Oct 30 21:54:10.406267 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:10.402280    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Oct 30 21:54:10.406267 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:10.402352    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj" podUID="d9a09763-b456-469d-80c7-a787456003f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Oct 30 21:54:11.402267 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:11.402205    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Oct 30 21:54:11.402475 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:11.402292    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj" podUID="d9a09763-b456-469d-80c7-a787456003f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Oct 30 21:54:12.417420 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:12.417378    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Oct 30 21:54:12.418082 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:12.418045    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj" podUID="d9a09763-b456-469d-80c7-a787456003f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Oct 30 21:54:12.452039 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:12.452004    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:12.615257 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:12.613925    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:13.401820 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:13.401772    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Oct 30 21:54:13.402040 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:13.401844    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj" podUID="d9a09763-b456-469d-80c7-a787456003f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Oct 30 21:54:14.527432 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:14.527393    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"
Oct 30 21:54:14.527883 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:14.527699    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"
Oct 30 21:54:14.528867 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:14.528847    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"
Oct 30 21:54:17.273811 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:17.273766    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:22.116926 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:22.116879    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:26.923091 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:26.917938    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:31.868980 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:31.868942    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:33.583098 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:33.583058    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:34.841415 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:34.841369    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:35.037497 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:35.037451    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:35.571258 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:54:35.570789    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:54:36.576005 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:36.575968    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:41.097831 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:41.097800    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:41.296022 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:41.295988    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:42.366191 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:42.366155    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:42.565323 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:42.565286    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:46.094087 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:46.094051    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:46.810864 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:46.810826    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:50.775209 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:50.774995    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:55.609002 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:54:55.608959    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:00.218074 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:00.218040    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:04.829704 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:04.829661    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:05.569028 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:55:05.568987    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:55:09.407680 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:09.407634    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:12.421835 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:12.421787    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:14.168144 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:14.168108    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:15.024981 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:15.024944    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"]
Oct 30 21:55:15.029358 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:15.029333    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"
Oct 30 21:55:15.112177 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:15.112128    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffc32abd-61d0-447a-9050-2ad518ca585f-istio-data\") pod \"maas-default-gateway-openshift-default-7df854bf67-btfp8\" (UID: \"ffc32abd-61d0-447a-9050-2ad518ca585f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"
Oct 30 21:55:15.112177 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:15.112174    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffc32abd-61d0-447a-9050-2ad518ca585f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7df854bf67-btfp8\" (UID: \"ffc32abd-61d0-447a-9050-2ad518ca585f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"
Oct 30 21:55:15.112451 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:15.112326    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffc32abd-61d0-447a-9050-2ad518ca585f-workload-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-btfp8\" (UID: \"ffc32abd-61d0-447a-9050-2ad518ca585f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"
Oct 30 21:55:15.112451 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:15.112407    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffc32abd-61d0-447a-9050-2ad518ca585f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7df854bf67-btfp8\" (UID: \"ffc32abd-61d0-447a-9050-2ad518ca585f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"
Oct 30 21:55:15.112566 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:15.112470    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffc32abd-61d0-447a-9050-2ad518ca585f-credential-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-btfp8\" (UID: \"ffc32abd-61d0-447a-9050-2ad518ca585f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"
Oct 30 21:55:15.112566 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:15.112538    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffc32abd-61d0-447a-9050-2ad518ca585f-workload-certs\") pod \"maas-default-gateway-openshift-default-7df854bf67-btfp8\" (UID: \"ffc32abd-61d0-447a-9050-2ad518ca585f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"
Oct 30 21:55:15.112671 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:15.112599    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffc32abd-61d0-447a-9050-2ad518ca585f-istio-token\") pod \"maas-default-gateway-openshift-default-7df854bf67-btfp8\" (UID: \"ffc32abd-61d0-447a-9050-2ad518ca585f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"
Oct 30 21:55:15.112671 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:15.112630    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tghmq\" (UniqueName: \"kubernetes.io/projected/ffc32abd-61d0-447a-9050-2ad518ca585f-kube-api-access-tghmq\") pod \"maas-default-gateway-openshift-default-7df854bf67-btfp8\" (UID: \"ffc32abd-61d0-447a-9050-2ad518ca585f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"
Oct 30 21:55:15.112763 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:15.112686    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffc32abd-61d0-447a-9050-2ad518ca585f-istio-envoy\") pod \"maas-default-gateway-openshift-default-7df854bf67-btfp8\" (UID: \"ffc32abd-61d0-447a-9050-2ad518ca585f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"
Oct 30 21:55:15.127041 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:15.127004    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"]
Oct 30 21:55:15.214874 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:15.213672    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffc32abd-61d0-447a-9050-2ad518ca585f-istio-token\") pod \"maas-default-gateway-openshift-default-7df854bf67-btfp8\" (UID: \"ffc32abd-61d0-447a-9050-2ad518ca585f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"
Oct 30 21:55:15.214874 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:15.213734    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tghmq\" (UniqueName: \"kubernetes.io/projected/ffc32abd-61d0-447a-9050-2ad518ca585f-kube-api-access-tghmq\") pod \"maas-default-gateway-openshift-default-7df854bf67-btfp8\" (UID: \"ffc32abd-61d0-447a-9050-2ad518ca585f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"
Oct 30 21:55:15.214874 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:15.213798    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffc32abd-61d0-447a-9050-2ad518ca585f-istio-envoy\") pod \"maas-default-gateway-openshift-default-7df854bf67-btfp8\" (UID: \"ffc32abd-61d0-447a-9050-2ad518ca585f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"
Oct 30 21:55:15.214874 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:15.213841    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffc32abd-61d0-447a-9050-2ad518ca585f-istio-data\") pod \"maas-default-gateway-openshift-default-7df854bf67-btfp8\" (UID: \"ffc32abd-61d0-447a-9050-2ad518ca585f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"
Oct 30 21:55:15.214874 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:15.213871    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffc32abd-61d0-447a-9050-2ad518ca585f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7df854bf67-btfp8\" (UID: \"ffc32abd-61d0-447a-9050-2ad518ca585f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"
Oct 30 21:55:15.214874 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:15.213953    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffc32abd-61d0-447a-9050-2ad518ca585f-workload-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-btfp8\" (UID: \"ffc32abd-61d0-447a-9050-2ad518ca585f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"
Oct 30 21:55:15.214874 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:15.214004    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffc32abd-61d0-447a-9050-2ad518ca585f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7df854bf67-btfp8\" (UID: \"ffc32abd-61d0-447a-9050-2ad518ca585f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"
Oct 30 21:55:15.214874 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:15.214108    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffc32abd-61d0-447a-9050-2ad518ca585f-credential-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-btfp8\" (UID: \"ffc32abd-61d0-447a-9050-2ad518ca585f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"
Oct 30 21:55:15.214874 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:15.214135    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffc32abd-61d0-447a-9050-2ad518ca585f-workload-certs\") pod \"maas-default-gateway-openshift-default-7df854bf67-btfp8\" (UID: \"ffc32abd-61d0-447a-9050-2ad518ca585f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"
Oct 30 21:55:15.214874 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:15.214531    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffc32abd-61d0-447a-9050-2ad518ca585f-workload-certs\") pod \"maas-default-gateway-openshift-default-7df854bf67-btfp8\" (UID: \"ffc32abd-61d0-447a-9050-2ad518ca585f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"
Oct 30 21:55:15.215679 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:15.214980    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffc32abd-61d0-447a-9050-2ad518ca585f-workload-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-btfp8\" (UID: \"ffc32abd-61d0-447a-9050-2ad518ca585f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"
Oct 30 21:55:15.215679 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:15.215015    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffc32abd-61d0-447a-9050-2ad518ca585f-istio-data\") pod \"maas-default-gateway-openshift-default-7df854bf67-btfp8\" (UID: \"ffc32abd-61d0-447a-9050-2ad518ca585f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"
Oct 30 21:55:15.215679 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:15.215353    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffc32abd-61d0-447a-9050-2ad518ca585f-credential-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-btfp8\" (UID: \"ffc32abd-61d0-447a-9050-2ad518ca585f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"
Oct 30 21:55:15.215679 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:15.215394    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffc32abd-61d0-447a-9050-2ad518ca585f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7df854bf67-btfp8\" (UID: \"ffc32abd-61d0-447a-9050-2ad518ca585f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"
Oct 30 21:55:15.216677 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:15.216652    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffc32abd-61d0-447a-9050-2ad518ca585f-istio-envoy\") pod \"maas-default-gateway-openshift-default-7df854bf67-btfp8\" (UID: \"ffc32abd-61d0-447a-9050-2ad518ca585f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"
Oct 30 21:55:15.219330 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:15.219306    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffc32abd-61d0-447a-9050-2ad518ca585f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7df854bf67-btfp8\" (UID: \"ffc32abd-61d0-447a-9050-2ad518ca585f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"
Oct 30 21:55:15.272562 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:15.272523    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffc32abd-61d0-447a-9050-2ad518ca585f-istio-token\") pod \"maas-default-gateway-openshift-default-7df854bf67-btfp8\" (UID: \"ffc32abd-61d0-447a-9050-2ad518ca585f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"
Oct 30 21:55:15.272854 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:15.272828    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tghmq\" (UniqueName: \"kubernetes.io/projected/ffc32abd-61d0-447a-9050-2ad518ca585f-kube-api-access-tghmq\") pod \"maas-default-gateway-openshift-default-7df854bf67-btfp8\" (UID: \"ffc32abd-61d0-447a-9050-2ad518ca585f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"
Oct 30 21:55:15.343196 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:15.343090    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"
Oct 30 21:55:15.767507 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:15.767479    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"]
Oct 30 21:55:15.769924 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:55:15.769887    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffc32abd_61d0_447a_9050_2ad518ca585f.slice/crio-42c90767c3085aacba721ec56da1b4ac3cd5434b541c38c1a1a539c96d2000b2 WatchSource:0}: Error finding container 42c90767c3085aacba721ec56da1b4ac3cd5434b541c38c1a1a539c96d2000b2: Status 404 returned error can't find the container with id 42c90767c3085aacba721ec56da1b4ac3cd5434b541c38c1a1a539c96d2000b2
Oct 30 21:55:15.772459 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:15.772428    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:55:15.772539 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:15.772497    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:55:15.772539 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:15.772526    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:55:16.360047 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:16.360004    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8" event={"ID":"ffc32abd-61d0-447a-9050-2ad518ca585f","Type":"ContainerStarted","Data":"74b1a584476cfaea2631fc7f42dd12b0cf532676d1098a88eb9a215d8bb7905b"}
Oct 30 21:55:16.360524 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:16.360072    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8" event={"ID":"ffc32abd-61d0-447a-9050-2ad518ca585f","Type":"ContainerStarted","Data":"42c90767c3085aacba721ec56da1b4ac3cd5434b541c38c1a1a539c96d2000b2"}
Oct 30 21:55:16.494540 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:16.494483    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8" podStartSLOduration=2.494467107 podStartE2EDuration="2.494467107s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:55:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:55:16.491980393 +0000 UTC m=+1058.669469010" watchObservedRunningTime="2026-10-30 21:55:16.494467107 +0000 UTC m=+1058.671955722"
Oct 30 21:55:17.343498 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:17.343461    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"
Oct 30 21:55:17.344899 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:17.344870    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-btfp8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:55:17.345052 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:17.344937    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8" podUID="ffc32abd-61d0-447a-9050-2ad518ca585f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 21:55:18.343830 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:18.343789    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-btfp8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:55:18.344277 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:18.343852    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8" podUID="ffc32abd-61d0-447a-9050-2ad518ca585f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 21:55:18.739251 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:18.739210    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:19.343790 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:19.343747    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-btfp8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:55:19.343980 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:19.343836    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8" podUID="ffc32abd-61d0-447a-9050-2ad518ca585f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 21:55:20.343822 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:20.343790    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-btfp8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:55:20.344011 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:20.343849    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8" podUID="ffc32abd-61d0-447a-9050-2ad518ca585f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 21:55:21.344053 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:21.344011    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-btfp8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:55:21.344497 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:21.344091    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8" podUID="ffc32abd-61d0-447a-9050-2ad518ca585f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 21:55:22.343454 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:22.343410    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-btfp8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:55:22.343669 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:22.343490    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8" podUID="ffc32abd-61d0-447a-9050-2ad518ca585f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 21:55:23.343740 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:23.343700    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-btfp8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:55:23.344403 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:23.344365    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8" podUID="ffc32abd-61d0-447a-9050-2ad518ca585f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 21:55:23.403960 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:23.403914    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:24.675551 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:24.675518    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"
Oct 30 21:55:24.676227 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:24.676192    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"
Oct 30 21:55:24.677650 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:24.677624    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"
Oct 30 21:55:27.429586 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:27.429551    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-64f8c8bfbd-l9dfg"]
Oct 30 21:55:27.433928 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:27.433902    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-64f8c8bfbd-l9dfg"
Oct 30 21:55:27.531710 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:27.531667    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"routing-overlay\""
Oct 30 21:55:27.531911 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:27.531775    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-plugins\""
Oct 30 21:55:27.531911 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:27.531791    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"openshift-service-ca.crt\""
Oct 30 21:55:27.531911 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:27.531801    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"kube-root-ca.crt\""
Oct 30 21:55:27.531911 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:27.531775    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-fd60520bcb62\""
Oct 30 21:55:27.532690 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:27.532676    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-dockercfg-pjjkg\""
Oct 30 21:55:27.535979 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:27.535957    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-64f8c8bfbd-l9dfg"]
Oct 30 21:55:27.556075 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:27.556041    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/0b56da79-7e87-41b3-9a07-e320389b9833-routing-overlay-volume\") pod \"payload-processing-external-model-64f8c8bfbd-l9dfg\" (UID: \"0b56da79-7e87-41b3-9a07-e320389b9833\") " pod="models-as-a-service/payload-processing-external-model-64f8c8bfbd-l9dfg"
Oct 30 21:55:27.556259 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:27.556103    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjq82\" (UniqueName: \"kubernetes.io/projected/0b56da79-7e87-41b3-9a07-e320389b9833-kube-api-access-zjq82\") pod \"payload-processing-external-model-64f8c8bfbd-l9dfg\" (UID: \"0b56da79-7e87-41b3-9a07-e320389b9833\") " pod="models-as-a-service/payload-processing-external-model-64f8c8bfbd-l9dfg"
Oct 30 21:55:27.556259 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:27.556201    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/0b56da79-7e87-41b3-9a07-e320389b9833-provider-credentials\") pod \"payload-processing-external-model-64f8c8bfbd-l9dfg\" (UID: \"0b56da79-7e87-41b3-9a07-e320389b9833\") " pod="models-as-a-service/payload-processing-external-model-64f8c8bfbd-l9dfg"
Oct 30 21:55:27.556364 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:27.556328    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b56da79-7e87-41b3-9a07-e320389b9833-plugins-config-volume\") pod \"payload-processing-external-model-64f8c8bfbd-l9dfg\" (UID: \"0b56da79-7e87-41b3-9a07-e320389b9833\") " pod="models-as-a-service/payload-processing-external-model-64f8c8bfbd-l9dfg"
Oct 30 21:55:27.657366 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:27.657324    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/0b56da79-7e87-41b3-9a07-e320389b9833-routing-overlay-volume\") pod \"payload-processing-external-model-64f8c8bfbd-l9dfg\" (UID: \"0b56da79-7e87-41b3-9a07-e320389b9833\") " pod="models-as-a-service/payload-processing-external-model-64f8c8bfbd-l9dfg"
Oct 30 21:55:27.657545 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:27.657483    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjq82\" (UniqueName: \"kubernetes.io/projected/0b56da79-7e87-41b3-9a07-e320389b9833-kube-api-access-zjq82\") pod \"payload-processing-external-model-64f8c8bfbd-l9dfg\" (UID: \"0b56da79-7e87-41b3-9a07-e320389b9833\") " pod="models-as-a-service/payload-processing-external-model-64f8c8bfbd-l9dfg"
Oct 30 21:55:27.657545 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:27.657521    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/0b56da79-7e87-41b3-9a07-e320389b9833-provider-credentials\") pod \"payload-processing-external-model-64f8c8bfbd-l9dfg\" (UID: \"0b56da79-7e87-41b3-9a07-e320389b9833\") " pod="models-as-a-service/payload-processing-external-model-64f8c8bfbd-l9dfg"
Oct 30 21:55:27.657626 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:27.657582    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b56da79-7e87-41b3-9a07-e320389b9833-plugins-config-volume\") pod \"payload-processing-external-model-64f8c8bfbd-l9dfg\" (UID: \"0b56da79-7e87-41b3-9a07-e320389b9833\") " pod="models-as-a-service/payload-processing-external-model-64f8c8bfbd-l9dfg"
Oct 30 21:55:27.658059 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:27.658036    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/0b56da79-7e87-41b3-9a07-e320389b9833-routing-overlay-volume\") pod \"payload-processing-external-model-64f8c8bfbd-l9dfg\" (UID: \"0b56da79-7e87-41b3-9a07-e320389b9833\") " pod="models-as-a-service/payload-processing-external-model-64f8c8bfbd-l9dfg"
Oct 30 21:55:27.658339 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:27.658321    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b56da79-7e87-41b3-9a07-e320389b9833-plugins-config-volume\") pod \"payload-processing-external-model-64f8c8bfbd-l9dfg\" (UID: \"0b56da79-7e87-41b3-9a07-e320389b9833\") " pod="models-as-a-service/payload-processing-external-model-64f8c8bfbd-l9dfg"
Oct 30 21:55:27.659993 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:27.659961    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/0b56da79-7e87-41b3-9a07-e320389b9833-provider-credentials\") pod \"payload-processing-external-model-64f8c8bfbd-l9dfg\" (UID: \"0b56da79-7e87-41b3-9a07-e320389b9833\") " pod="models-as-a-service/payload-processing-external-model-64f8c8bfbd-l9dfg"
Oct 30 21:55:27.715840 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:27.715759    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjq82\" (UniqueName: \"kubernetes.io/projected/0b56da79-7e87-41b3-9a07-e320389b9833-kube-api-access-zjq82\") pod \"payload-processing-external-model-64f8c8bfbd-l9dfg\" (UID: \"0b56da79-7e87-41b3-9a07-e320389b9833\") " pod="models-as-a-service/payload-processing-external-model-64f8c8bfbd-l9dfg"
Oct 30 21:55:27.759036 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:27.758997    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-64f8c8bfbd-l9dfg"
Oct 30 21:55:28.130684 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:28.130651    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:28.142835 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:28.142796    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-64f8c8bfbd-l9dfg"]
Oct 30 21:55:28.469432 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:28.469329    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-64f8c8bfbd-l9dfg" event={"ID":"0b56da79-7e87-41b3-9a07-e320389b9833","Type":"ContainerStarted","Data":"c2f1bb98a5a3f41689c3a33ba266bbdf151e2e30f6792901622878a88c03baaf"}
Oct 30 21:55:28.469432 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:28.469405    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-64f8c8bfbd-l9dfg" event={"ID":"0b56da79-7e87-41b3-9a07-e320389b9833","Type":"ContainerStarted","Data":"6c8c2a8de2f66db2e4854f6a2274c72e4d7769fcf13d3539997bc79b3d6a47ea"}
Oct 30 21:55:28.469913 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:28.469497    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-64f8c8bfbd-l9dfg"
Oct 30 21:55:28.580069 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:28.580008    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-64f8c8bfbd-l9dfg" podStartSLOduration=1.579990396 podStartE2EDuration="1.579990396s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:55:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:55:28.579836777 +0000 UTC m=+1070.757325394" watchObservedRunningTime="2026-10-30 21:55:28.579990396 +0000 UTC m=+1070.757479013"
Oct 30 21:55:30.018833 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.018791    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"]
Oct 30 21:55:30.023184 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.023164    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"
Oct 30 21:55:30.064621 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.064584    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"]
Oct 30 21:55:30.068456 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.068434    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"
Oct 30 21:55:30.086104 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.086075    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b28e1c99-b48d-4a4e-a225-ebba779327e4-istio-envoy\") pod \"maas-default-gateway-openshift-default-7df854bf67-bpm9h\" (UID: \"b28e1c99-b48d-4a4e-a225-ebba779327e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"
Oct 30 21:55:30.086104 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.086107    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c48rc\" (UniqueName: \"kubernetes.io/projected/b28e1c99-b48d-4a4e-a225-ebba779327e4-kube-api-access-c48rc\") pod \"maas-default-gateway-openshift-default-7df854bf67-bpm9h\" (UID: \"b28e1c99-b48d-4a4e-a225-ebba779327e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"
Oct 30 21:55:30.086364 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.086136    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b28e1c99-b48d-4a4e-a225-ebba779327e4-workload-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-bpm9h\" (UID: \"b28e1c99-b48d-4a4e-a225-ebba779327e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"
Oct 30 21:55:30.086364 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.086162    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b28e1c99-b48d-4a4e-a225-ebba779327e4-workload-certs\") pod \"maas-default-gateway-openshift-default-7df854bf67-bpm9h\" (UID: \"b28e1c99-b48d-4a4e-a225-ebba779327e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"
Oct 30 21:55:30.086364 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.086339    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b28e1c99-b48d-4a4e-a225-ebba779327e4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7df854bf67-bpm9h\" (UID: \"b28e1c99-b48d-4a4e-a225-ebba779327e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"
Oct 30 21:55:30.086535 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.086404    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b28e1c99-b48d-4a4e-a225-ebba779327e4-credential-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-bpm9h\" (UID: \"b28e1c99-b48d-4a4e-a225-ebba779327e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"
Oct 30 21:55:30.086535 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.086444    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b28e1c99-b48d-4a4e-a225-ebba779327e4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7df854bf67-bpm9h\" (UID: \"b28e1c99-b48d-4a4e-a225-ebba779327e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"
Oct 30 21:55:30.086535 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.086470    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b28e1c99-b48d-4a4e-a225-ebba779327e4-istio-token\") pod \"maas-default-gateway-openshift-default-7df854bf67-bpm9h\" (UID: \"b28e1c99-b48d-4a4e-a225-ebba779327e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"
Oct 30 21:55:30.086535 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.086522    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b28e1c99-b48d-4a4e-a225-ebba779327e4-istio-data\") pod \"maas-default-gateway-openshift-default-7df854bf67-bpm9h\" (UID: \"b28e1c99-b48d-4a4e-a225-ebba779327e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"
Oct 30 21:55:30.119320 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.119277    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"]
Oct 30 21:55:30.187659 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.187615    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19911666-f25a-4dd7-9789-7b9374b61cfb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7df854bf67-mrq5j\" (UID: \"19911666-f25a-4dd7-9789-7b9374b61cfb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"
Oct 30 21:55:30.187878 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.187676    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwqrc\" (UniqueName: \"kubernetes.io/projected/19911666-f25a-4dd7-9789-7b9374b61cfb-kube-api-access-bwqrc\") pod \"maas-default-gateway-openshift-default-7df854bf67-mrq5j\" (UID: \"19911666-f25a-4dd7-9789-7b9374b61cfb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"
Oct 30 21:55:30.187878 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.187750    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b28e1c99-b48d-4a4e-a225-ebba779327e4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7df854bf67-bpm9h\" (UID: \"b28e1c99-b48d-4a4e-a225-ebba779327e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"
Oct 30 21:55:30.187878 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.187784    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19911666-f25a-4dd7-9789-7b9374b61cfb-istio-data\") pod \"maas-default-gateway-openshift-default-7df854bf67-mrq5j\" (UID: \"19911666-f25a-4dd7-9789-7b9374b61cfb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"
Oct 30 21:55:30.187878 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.187825    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b28e1c99-b48d-4a4e-a225-ebba779327e4-credential-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-bpm9h\" (UID: \"b28e1c99-b48d-4a4e-a225-ebba779327e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"
Oct 30 21:55:30.188099 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.187893    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19911666-f25a-4dd7-9789-7b9374b61cfb-credential-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-mrq5j\" (UID: \"19911666-f25a-4dd7-9789-7b9374b61cfb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"
Oct 30 21:55:30.188099 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.187918    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19911666-f25a-4dd7-9789-7b9374b61cfb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7df854bf67-mrq5j\" (UID: \"19911666-f25a-4dd7-9789-7b9374b61cfb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"
Oct 30 21:55:30.188099 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.187939    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b28e1c99-b48d-4a4e-a225-ebba779327e4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7df854bf67-bpm9h\" (UID: \"b28e1c99-b48d-4a4e-a225-ebba779327e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"
Oct 30 21:55:30.188099 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.187960    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b28e1c99-b48d-4a4e-a225-ebba779327e4-istio-token\") pod \"maas-default-gateway-openshift-default-7df854bf67-bpm9h\" (UID: \"b28e1c99-b48d-4a4e-a225-ebba779327e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"
Oct 30 21:55:30.188099 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.187981    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19911666-f25a-4dd7-9789-7b9374b61cfb-istio-token\") pod \"maas-default-gateway-openshift-default-7df854bf67-mrq5j\" (UID: \"19911666-f25a-4dd7-9789-7b9374b61cfb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"
Oct 30 21:55:30.188099 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.188001    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b28e1c99-b48d-4a4e-a225-ebba779327e4-istio-data\") pod \"maas-default-gateway-openshift-default-7df854bf67-bpm9h\" (UID: \"b28e1c99-b48d-4a4e-a225-ebba779327e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"
Oct 30 21:55:30.188099 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.188036    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19911666-f25a-4dd7-9789-7b9374b61cfb-istio-envoy\") pod \"maas-default-gateway-openshift-default-7df854bf67-mrq5j\" (UID: \"19911666-f25a-4dd7-9789-7b9374b61cfb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"
Oct 30 21:55:30.188099 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.188087    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b28e1c99-b48d-4a4e-a225-ebba779327e4-istio-envoy\") pod \"maas-default-gateway-openshift-default-7df854bf67-bpm9h\" (UID: \"b28e1c99-b48d-4a4e-a225-ebba779327e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"
Oct 30 21:55:30.188099 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.188103    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c48rc\" (UniqueName: \"kubernetes.io/projected/b28e1c99-b48d-4a4e-a225-ebba779327e4-kube-api-access-c48rc\") pod \"maas-default-gateway-openshift-default-7df854bf67-bpm9h\" (UID: \"b28e1c99-b48d-4a4e-a225-ebba779327e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"
Oct 30 21:55:30.188591 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.188123    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19911666-f25a-4dd7-9789-7b9374b61cfb-workload-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-mrq5j\" (UID: \"19911666-f25a-4dd7-9789-7b9374b61cfb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"
Oct 30 21:55:30.188591 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.188151    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b28e1c99-b48d-4a4e-a225-ebba779327e4-workload-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-bpm9h\" (UID: \"b28e1c99-b48d-4a4e-a225-ebba779327e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"
Oct 30 21:55:30.188591 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.188172    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b28e1c99-b48d-4a4e-a225-ebba779327e4-workload-certs\") pod \"maas-default-gateway-openshift-default-7df854bf67-bpm9h\" (UID: \"b28e1c99-b48d-4a4e-a225-ebba779327e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"
Oct 30 21:55:30.188591 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.188186    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19911666-f25a-4dd7-9789-7b9374b61cfb-workload-certs\") pod \"maas-default-gateway-openshift-default-7df854bf67-mrq5j\" (UID: \"19911666-f25a-4dd7-9789-7b9374b61cfb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"
Oct 30 21:55:30.188591 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.188528    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b28e1c99-b48d-4a4e-a225-ebba779327e4-credential-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-bpm9h\" (UID: \"b28e1c99-b48d-4a4e-a225-ebba779327e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"
Oct 30 21:55:30.188901 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.188678    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b28e1c99-b48d-4a4e-a225-ebba779327e4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7df854bf67-bpm9h\" (UID: \"b28e1c99-b48d-4a4e-a225-ebba779327e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"
Oct 30 21:55:30.189221 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.189156    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b28e1c99-b48d-4a4e-a225-ebba779327e4-workload-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-bpm9h\" (UID: \"b28e1c99-b48d-4a4e-a225-ebba779327e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"
Oct 30 21:55:30.189421 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.189396    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b28e1c99-b48d-4a4e-a225-ebba779327e4-istio-data\") pod \"maas-default-gateway-openshift-default-7df854bf67-bpm9h\" (UID: \"b28e1c99-b48d-4a4e-a225-ebba779327e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"
Oct 30 21:55:30.189602 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.189458    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b28e1c99-b48d-4a4e-a225-ebba779327e4-workload-certs\") pod \"maas-default-gateway-openshift-default-7df854bf67-bpm9h\" (UID: \"b28e1c99-b48d-4a4e-a225-ebba779327e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"
Oct 30 21:55:30.191038 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.191017    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b28e1c99-b48d-4a4e-a225-ebba779327e4-istio-envoy\") pod \"maas-default-gateway-openshift-default-7df854bf67-bpm9h\" (UID: \"b28e1c99-b48d-4a4e-a225-ebba779327e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"
Oct 30 21:55:30.191183 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.191077    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b28e1c99-b48d-4a4e-a225-ebba779327e4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7df854bf67-bpm9h\" (UID: \"b28e1c99-b48d-4a4e-a225-ebba779327e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"
Oct 30 21:55:30.216298 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.216265    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"]
Oct 30 21:55:30.250277 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.247948    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b28e1c99-b48d-4a4e-a225-ebba779327e4-istio-token\") pod \"maas-default-gateway-openshift-default-7df854bf67-bpm9h\" (UID: \"b28e1c99-b48d-4a4e-a225-ebba779327e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"
Oct 30 21:55:30.250277 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.248012    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c48rc\" (UniqueName: \"kubernetes.io/projected/b28e1c99-b48d-4a4e-a225-ebba779327e4-kube-api-access-c48rc\") pod \"maas-default-gateway-openshift-default-7df854bf67-bpm9h\" (UID: \"b28e1c99-b48d-4a4e-a225-ebba779327e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"
Oct 30 21:55:30.289314 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.289204    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19911666-f25a-4dd7-9789-7b9374b61cfb-workload-certs\") pod \"maas-default-gateway-openshift-default-7df854bf67-mrq5j\" (UID: \"19911666-f25a-4dd7-9789-7b9374b61cfb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"
Oct 30 21:55:30.289314 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.289287    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19911666-f25a-4dd7-9789-7b9374b61cfb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7df854bf67-mrq5j\" (UID: \"19911666-f25a-4dd7-9789-7b9374b61cfb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"
Oct 30 21:55:30.289314 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.289313    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwqrc\" (UniqueName: \"kubernetes.io/projected/19911666-f25a-4dd7-9789-7b9374b61cfb-kube-api-access-bwqrc\") pod \"maas-default-gateway-openshift-default-7df854bf67-mrq5j\" (UID: \"19911666-f25a-4dd7-9789-7b9374b61cfb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"
Oct 30 21:55:30.289609 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.289338    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19911666-f25a-4dd7-9789-7b9374b61cfb-istio-data\") pod \"maas-default-gateway-openshift-default-7df854bf67-mrq5j\" (UID: \"19911666-f25a-4dd7-9789-7b9374b61cfb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"
Oct 30 21:55:30.289609 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.289386    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19911666-f25a-4dd7-9789-7b9374b61cfb-credential-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-mrq5j\" (UID: \"19911666-f25a-4dd7-9789-7b9374b61cfb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"
Oct 30 21:55:30.289609 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.289405    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19911666-f25a-4dd7-9789-7b9374b61cfb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7df854bf67-mrq5j\" (UID: \"19911666-f25a-4dd7-9789-7b9374b61cfb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"
Oct 30 21:55:30.289609 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.289441    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19911666-f25a-4dd7-9789-7b9374b61cfb-istio-token\") pod \"maas-default-gateway-openshift-default-7df854bf67-mrq5j\" (UID: \"19911666-f25a-4dd7-9789-7b9374b61cfb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"
Oct 30 21:55:30.289609 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.289482    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19911666-f25a-4dd7-9789-7b9374b61cfb-istio-envoy\") pod \"maas-default-gateway-openshift-default-7df854bf67-mrq5j\" (UID: \"19911666-f25a-4dd7-9789-7b9374b61cfb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"
Oct 30 21:55:30.289837 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.289683    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19911666-f25a-4dd7-9789-7b9374b61cfb-workload-certs\") pod \"maas-default-gateway-openshift-default-7df854bf67-mrq5j\" (UID: \"19911666-f25a-4dd7-9789-7b9374b61cfb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"
Oct 30 21:55:30.289837 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.289712    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19911666-f25a-4dd7-9789-7b9374b61cfb-istio-data\") pod \"maas-default-gateway-openshift-default-7df854bf67-mrq5j\" (UID: \"19911666-f25a-4dd7-9789-7b9374b61cfb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"
Oct 30 21:55:30.289837 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.289706    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19911666-f25a-4dd7-9789-7b9374b61cfb-workload-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-mrq5j\" (UID: \"19911666-f25a-4dd7-9789-7b9374b61cfb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"
Oct 30 21:55:30.290069 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.290049    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19911666-f25a-4dd7-9789-7b9374b61cfb-credential-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-mrq5j\" (UID: \"19911666-f25a-4dd7-9789-7b9374b61cfb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"
Oct 30 21:55:30.290156 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.290136    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19911666-f25a-4dd7-9789-7b9374b61cfb-workload-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-mrq5j\" (UID: \"19911666-f25a-4dd7-9789-7b9374b61cfb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"
Oct 30 21:55:30.290374 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.290355    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19911666-f25a-4dd7-9789-7b9374b61cfb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7df854bf67-mrq5j\" (UID: \"19911666-f25a-4dd7-9789-7b9374b61cfb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"
Oct 30 21:55:30.291808 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.291786    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19911666-f25a-4dd7-9789-7b9374b61cfb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7df854bf67-mrq5j\" (UID: \"19911666-f25a-4dd7-9789-7b9374b61cfb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"
Oct 30 21:55:30.291899 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.291787    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19911666-f25a-4dd7-9789-7b9374b61cfb-istio-envoy\") pod \"maas-default-gateway-openshift-default-7df854bf67-mrq5j\" (UID: \"19911666-f25a-4dd7-9789-7b9374b61cfb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"
Oct 30 21:55:30.338157 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.338118    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"
Oct 30 21:55:30.347704 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.347675    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19911666-f25a-4dd7-9789-7b9374b61cfb-istio-token\") pod \"maas-default-gateway-openshift-default-7df854bf67-mrq5j\" (UID: \"19911666-f25a-4dd7-9789-7b9374b61cfb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"
Oct 30 21:55:30.347704 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.347689    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwqrc\" (UniqueName: \"kubernetes.io/projected/19911666-f25a-4dd7-9789-7b9374b61cfb-kube-api-access-bwqrc\") pod \"maas-default-gateway-openshift-default-7df854bf67-mrq5j\" (UID: \"19911666-f25a-4dd7-9789-7b9374b61cfb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"
Oct 30 21:55:30.383416 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.382842    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"
Oct 30 21:55:30.767021 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.766985    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"]
Oct 30 21:55:30.767865 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:55:30.767838    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb28e1c99_b48d_4a4e_a225_ebba779327e4.slice/crio-98ac329ac724c42b57b6dac2f63e075039febe24ca20e5464552d3e08377d68b WatchSource:0}: Error finding container 98ac329ac724c42b57b6dac2f63e075039febe24ca20e5464552d3e08377d68b: Status 404 returned error can't find the container with id 98ac329ac724c42b57b6dac2f63e075039febe24ca20e5464552d3e08377d68b
Oct 30 21:55:30.770327 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.770300    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:55:30.770385 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.770367    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:55:30.770430 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.770398    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:55:30.818996 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.818961    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"]
Oct 30 21:55:30.833485 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:55:30.833455    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19911666_f25a_4dd7_9789_7b9374b61cfb.slice/crio-00c42bd448198dcc73803fdf7d05aff173eff79290426e5a67cf89f2ed3510c9 WatchSource:0}: Error finding container 00c42bd448198dcc73803fdf7d05aff173eff79290426e5a67cf89f2ed3510c9: Status 404 returned error can't find the container with id 00c42bd448198dcc73803fdf7d05aff173eff79290426e5a67cf89f2ed3510c9
Oct 30 21:55:30.836204 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.836158    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:55:30.836334 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.836284    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:55:30.836385 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:30.836339    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:55:31.491710 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:31.491678    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j" event={"ID":"19911666-f25a-4dd7-9789-7b9374b61cfb","Type":"ContainerStarted","Data":"af426aeb828c2148a3ba12ce18f7495dc345415ee0419868e346a97bb8a09a8f"}
Oct 30 21:55:31.492227 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:31.491735    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j" event={"ID":"19911666-f25a-4dd7-9789-7b9374b61cfb","Type":"ContainerStarted","Data":"00c42bd448198dcc73803fdf7d05aff173eff79290426e5a67cf89f2ed3510c9"}
Oct 30 21:55:31.493484 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:31.493459    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h" event={"ID":"b28e1c99-b48d-4a4e-a225-ebba779327e4","Type":"ContainerStarted","Data":"c4df035ff8ad9aa06e79ea2ff334fd4a2b3b149452677e5223043972868bed19"}
Oct 30 21:55:31.493574 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:31.493505    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h" event={"ID":"b28e1c99-b48d-4a4e-a225-ebba779327e4","Type":"ContainerStarted","Data":"98ac329ac724c42b57b6dac2f63e075039febe24ca20e5464552d3e08377d68b"}
Oct 30 21:55:31.791490 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:31.791373    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j" podStartSLOduration=1.791357658 podStartE2EDuration="1.791357658s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:55:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:55:31.655672846 +0000 UTC m=+1073.833161466" watchObservedRunningTime="2026-10-30 21:55:31.791357658 +0000 UTC m=+1073.968846274"
Oct 30 21:55:31.792564 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:31.792528    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h" podStartSLOduration=2.792515132 podStartE2EDuration="2.792515132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:55:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:55:31.78865702 +0000 UTC m=+1073.966145638" watchObservedRunningTime="2026-10-30 21:55:31.792515132 +0000 UTC m=+1073.970003749"
Oct 30 21:55:32.352341 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:32.350389    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"
Oct 30 21:55:32.352341 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:32.350529    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-bpm9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 30 21:55:32.352341 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:32.350575    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h" podUID="b28e1c99-b48d-4a4e-a225-ebba779327e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 30 21:55:32.383635 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:32.383597    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"
Oct 30 21:55:32.384833 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:32.384808    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-mrq5j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 21:55:32.384944 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:32.384858    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j" podUID="19911666-f25a-4dd7-9789-7b9374b61cfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 21:55:32.779984 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:32.779880    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:33.338698 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:33.338652    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-bpm9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 30 21:55:33.338993 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:33.338962    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h" podUID="b28e1c99-b48d-4a4e-a225-ebba779327e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 30 21:55:33.384000 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:33.383966    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-mrq5j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 21:55:33.384301 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:33.384218    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j" podUID="19911666-f25a-4dd7-9789-7b9374b61cfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 21:55:33.892939 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:33.892908    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:34.349830 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:34.349731    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-bpm9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 30 21:55:34.350111 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:34.350079    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h" podUID="b28e1c99-b48d-4a4e-a225-ebba779327e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 30 21:55:34.384751 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:34.384717    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-mrq5j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 21:55:34.384969 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:34.384950    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j" podUID="19911666-f25a-4dd7-9789-7b9374b61cfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 21:55:35.339263 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:35.338802    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-bpm9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 30 21:55:35.339263 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:35.338904    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h" podUID="b28e1c99-b48d-4a4e-a225-ebba779327e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 30 21:55:35.384345 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:35.384303    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-mrq5j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 21:55:35.384621 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:35.384592    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j" podUID="19911666-f25a-4dd7-9789-7b9374b61cfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 21:55:35.599871 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:55:35.599792    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:55:36.339264 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:36.338745    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-bpm9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 30 21:55:36.339264 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:36.338828    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h" podUID="b28e1c99-b48d-4a4e-a225-ebba779327e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 30 21:55:36.384052 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:36.384013    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-mrq5j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 21:55:36.384249 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:36.384069    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j" podUID="19911666-f25a-4dd7-9789-7b9374b61cfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 21:55:37.338722 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:37.338675    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-bpm9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 30 21:55:37.338938 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:37.338748    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h" podUID="b28e1c99-b48d-4a4e-a225-ebba779327e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 30 21:55:37.384284 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:37.384216    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-mrq5j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 21:55:37.384678 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:37.384316    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j" podUID="19911666-f25a-4dd7-9789-7b9374b61cfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 21:55:37.623741 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:37.623696    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:38.338870 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:38.338817    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-bpm9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 30 21:55:38.339098 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:38.338900    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h" podUID="b28e1c99-b48d-4a4e-a225-ebba779327e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 30 21:55:38.384691 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:38.384646    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-mrq5j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 21:55:38.385158 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:38.384720    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j" podUID="19911666-f25a-4dd7-9789-7b9374b61cfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 21:55:39.480563 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:39.480507    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-64f8c8bfbd-l9dfg" podUID="0b56da79-7e87-41b3-9a07-e320389b9833" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:55:40.382432 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:40.382385    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-bpm9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:55:40.382778 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:40.382458    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h" podUID="b28e1c99-b48d-4a4e-a225-ebba779327e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:40.427405 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:40.427357    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-mrq5j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:55:40.427698 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:40.427663    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j" podUID="19911666-f25a-4dd7-9789-7b9374b61cfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:40.486066 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:40.486035    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"
Oct 30 21:55:40.487877 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:40.487853    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"
Oct 30 21:55:40.491333 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:40.491307    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"
Oct 30 21:55:40.534612 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:40.534578    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"
Oct 30 21:55:40.571259 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:40.536109    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"
Oct 30 21:55:40.574161 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:40.574133    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"
Oct 30 21:55:42.554266 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:42.546061    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:47.195491 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:47.195454    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:49.477054 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:49.477015    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-64f8c8bfbd-l9dfg" podUID="0b56da79-7e87-41b3-9a07-e320389b9833" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:55:51.796088 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:51.796050    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:56.418297 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:56.418218    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:57.995482 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:57.995444    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:59.477625 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:55:59.477583    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-64f8c8bfbd-l9dfg" podUID="0b56da79-7e87-41b3-9a07-e320389b9833" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:56:00.905977 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:00.905944    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:05.095202 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:05.095165    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:05.553715 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:05.553656    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:05.570575 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:56:05.570541    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:56:09.477408 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:09.477360    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-64f8c8bfbd-l9dfg" podUID="0b56da79-7e87-41b3-9a07-e320389b9833" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:56:10.221639 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:10.221602    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:14.821408 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:14.821359    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:19.477302 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:19.477200    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-64f8c8bfbd-l9dfg" podUID="0b56da79-7e87-41b3-9a07-e320389b9833" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:56:19.528054 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:19.528023    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:24.109785 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:24.109742    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:27.790318 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:27.790280    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"]
Oct 30 21:56:27.794903 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:27.794878    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"
Oct 30 21:56:27.844445 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:27.844408    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-9a5623-openshift-default-dockercfg-gltcf\""
Oct 30 21:56:27.897571 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:27.897533    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"]
Oct 30 21:56:27.900210 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:27.900175    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-istio-data\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k\" (UID: \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"
Oct 30 21:56:27.900379 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:27.900283    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-istio-token\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k\" (UID: \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"
Oct 30 21:56:27.900433 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:27.900416    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-istio-podinfo\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k\" (UID: \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"
Oct 30 21:56:27.900527 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:27.900505    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-credential-socket\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k\" (UID: \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"
Oct 30 21:56:27.900590 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:27.900577    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-istiod-ca-cert\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k\" (UID: \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"
Oct 30 21:56:27.900661 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:27.900644    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-workload-certs\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k\" (UID: \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"
Oct 30 21:56:27.900716 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:27.900688    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-workload-socket\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k\" (UID: \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"
Oct 30 21:56:27.900775 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:27.900761    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7n99\" (UniqueName: \"kubernetes.io/projected/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-kube-api-access-d7n99\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k\" (UID: \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"
Oct 30 21:56:27.900811 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:27.900786    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-istio-envoy\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k\" (UID: \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"
Oct 30 21:56:28.002455 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:28.002416    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-workload-certs\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k\" (UID: \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"
Oct 30 21:56:28.002664 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:28.002464    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-workload-socket\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k\" (UID: \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"
Oct 30 21:56:28.002664 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:28.002507    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7n99\" (UniqueName: \"kubernetes.io/projected/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-kube-api-access-d7n99\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k\" (UID: \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"
Oct 30 21:56:28.002664 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:28.002526    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-istio-envoy\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k\" (UID: \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"
Oct 30 21:56:28.002664 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:28.002569    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-istio-data\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k\" (UID: \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"
Oct 30 21:56:28.002664 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:28.002597    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-istio-token\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k\" (UID: \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"
Oct 30 21:56:28.002953 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:28.002682    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-istio-podinfo\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k\" (UID: \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"
Oct 30 21:56:28.002953 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:28.002711    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-credential-socket\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k\" (UID: \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"
Oct 30 21:56:28.002953 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:28.002741    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-istiod-ca-cert\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k\" (UID: \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"
Oct 30 21:56:28.002953 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:28.002898    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-workload-certs\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k\" (UID: \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"
Oct 30 21:56:28.002953 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:28.002935    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-workload-socket\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k\" (UID: \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"
Oct 30 21:56:28.003227 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:28.003190    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-istio-data\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k\" (UID: \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"
Oct 30 21:56:28.003376 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:28.003354    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-credential-socket\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k\" (UID: \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"
Oct 30 21:56:28.003701 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:28.003679    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-istiod-ca-cert\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k\" (UID: \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"
Oct 30 21:56:28.005251 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:28.005214    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-istio-envoy\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k\" (UID: \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"
Oct 30 21:56:28.005408 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:28.005389    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-istio-podinfo\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k\" (UID: \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"
Oct 30 21:56:28.061395 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:28.061295    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-istio-token\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k\" (UID: \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"
Oct 30 21:56:28.061395 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:28.061312    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7n99\" (UniqueName: \"kubernetes.io/projected/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-kube-api-access-d7n99\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k\" (UID: \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"
Oct 30 21:56:28.109225 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:28.109190    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"
Oct 30 21:56:28.547775 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:28.547741    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"]
Oct 30 21:56:28.549043 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:56:28.549006    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5b0cd90_fe64_4446_9a53_9ab9ebaa269e.slice/crio-9b5e481e1c830e69b97a9ef42a9261ebeaa2131940f78961eb4b416dde5e13c4 WatchSource:0}: Error finding container 9b5e481e1c830e69b97a9ef42a9261ebeaa2131940f78961eb4b416dde5e13c4: Status 404 returned error can't find the container with id 9b5e481e1c830e69b97a9ef42a9261ebeaa2131940f78961eb4b416dde5e13c4
Oct 30 21:56:28.551509 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:28.551479    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:56:28.551593 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:28.551545    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:56:28.551593 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:28.551573    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:56:28.716071 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:28.715998    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:29.055800 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:29.055756    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k" event={"ID":"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e","Type":"ContainerStarted","Data":"5684f813bd9bb13ddba214ca358f59f5f40f290556249dd8396053a96cba135f"}
Oct 30 21:56:29.056218 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:29.055823    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k" event={"ID":"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e","Type":"ContainerStarted","Data":"9b5e481e1c830e69b97a9ef42a9261ebeaa2131940f78961eb4b416dde5e13c4"}
Oct 30 21:56:29.111281 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:29.110645    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"
Oct 30 21:56:29.113059 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:29.113013    4871 patch_prober.go:29] interesting pod/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 21:56:29.113279 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:29.113145    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k" podUID="c5b0cd90-fe64-4446-9a53-9ab9ebaa269e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 21:56:29.175784 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:29.175707    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k" podStartSLOduration=2.17567999 podStartE2EDuration="2.17567999s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:56:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:56:29.17145146 +0000 UTC m=+1131.348940095" watchObservedRunningTime="2026-10-30 21:56:29.17567999 +0000 UTC m=+1131.353168607"
Oct 30 21:56:29.487584 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:29.487543    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-64f8c8bfbd-l9dfg" podUID="0b56da79-7e87-41b3-9a07-e320389b9833" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:56:29.952262 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:29.951403    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:30.116723 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:30.116686    4871 patch_prober.go:29] interesting pod/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 21:56:30.117361 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:30.117328    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k" podUID="c5b0cd90-fe64-4446-9a53-9ab9ebaa269e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 21:56:31.111257 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:31.110589    4871 patch_prober.go:29] interesting pod/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 21:56:31.111257 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:31.110657    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k" podUID="c5b0cd90-fe64-4446-9a53-9ab9ebaa269e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 21:56:32.111286 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:32.110715    4871 patch_prober.go:29] interesting pod/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 21:56:32.111286 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:32.110800    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k" podUID="c5b0cd90-fe64-4446-9a53-9ab9ebaa269e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 21:56:32.729447 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:32.729405    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-64f8c8bfbd-l9dfg"]
Oct 30 21:56:32.731602 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:32.731546    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-64f8c8bfbd-l9dfg" podUID="0b56da79-7e87-41b3-9a07-e320389b9833" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:56:32.754221 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:32.754191    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-866b9fd699-2z7wc"]
Oct 30 21:56:32.758910 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:32.758880    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-866b9fd699-2z7wc"
Oct 30 21:56:32.775976 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:32.775937    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnbgx\" (UniqueName: \"kubernetes.io/projected/2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f-kube-api-access-mnbgx\") pod \"payload-processing-external-model-866b9fd699-2z7wc\" (UID: \"2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f\") " pod="models-as-a-service/payload-processing-external-model-866b9fd699-2z7wc"
Oct 30 21:56:32.776270 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:32.776219    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f-routing-overlay-volume\") pod \"payload-processing-external-model-866b9fd699-2z7wc\" (UID: \"2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f\") " pod="models-as-a-service/payload-processing-external-model-866b9fd699-2z7wc"
Oct 30 21:56:32.776437 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:32.776419    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f-plugins-config-volume\") pod \"payload-processing-external-model-866b9fd699-2z7wc\" (UID: \"2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f\") " pod="models-as-a-service/payload-processing-external-model-866b9fd699-2z7wc"
Oct 30 21:56:32.776583 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:32.776566    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f-provider-credentials\") pod \"payload-processing-external-model-866b9fd699-2z7wc\" (UID: \"2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f\") " pod="models-as-a-service/payload-processing-external-model-866b9fd699-2z7wc"
Oct 30 21:56:32.816673 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:32.816641    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-7653478ffe43\""
Oct 30 21:56:32.852677 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:32.852590    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-866b9fd699-2z7wc"]
Oct 30 21:56:32.877819 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:32.877771    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnbgx\" (UniqueName: \"kubernetes.io/projected/2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f-kube-api-access-mnbgx\") pod \"payload-processing-external-model-866b9fd699-2z7wc\" (UID: \"2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f\") " pod="models-as-a-service/payload-processing-external-model-866b9fd699-2z7wc"
Oct 30 21:56:32.877994 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:32.877848    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f-routing-overlay-volume\") pod \"payload-processing-external-model-866b9fd699-2z7wc\" (UID: \"2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f\") " pod="models-as-a-service/payload-processing-external-model-866b9fd699-2z7wc"
Oct 30 21:56:32.877994 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:32.877897    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f-plugins-config-volume\") pod \"payload-processing-external-model-866b9fd699-2z7wc\" (UID: \"2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f\") " pod="models-as-a-service/payload-processing-external-model-866b9fd699-2z7wc"
Oct 30 21:56:32.877994 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:32.877937    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f-provider-credentials\") pod \"payload-processing-external-model-866b9fd699-2z7wc\" (UID: \"2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f\") " pod="models-as-a-service/payload-processing-external-model-866b9fd699-2z7wc"
Oct 30 21:56:32.878885 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:32.878836    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f-routing-overlay-volume\") pod \"payload-processing-external-model-866b9fd699-2z7wc\" (UID: \"2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f\") " pod="models-as-a-service/payload-processing-external-model-866b9fd699-2z7wc"
Oct 30 21:56:32.879539 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:32.879516    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f-plugins-config-volume\") pod \"payload-processing-external-model-866b9fd699-2z7wc\" (UID: \"2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f\") " pod="models-as-a-service/payload-processing-external-model-866b9fd699-2z7wc"
Oct 30 21:56:32.880695 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:32.880674    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f-provider-credentials\") pod \"payload-processing-external-model-866b9fd699-2z7wc\" (UID: \"2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f\") " pod="models-as-a-service/payload-processing-external-model-866b9fd699-2z7wc"
Oct 30 21:56:32.937755 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:32.937720    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnbgx\" (UniqueName: \"kubernetes.io/projected/2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f-kube-api-access-mnbgx\") pod \"payload-processing-external-model-866b9fd699-2z7wc\" (UID: \"2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f\") " pod="models-as-a-service/payload-processing-external-model-866b9fd699-2z7wc"
Oct 30 21:56:33.078913 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:33.078824    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-866b9fd699-2z7wc"
Oct 30 21:56:33.128261 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:33.126488    4871 patch_prober.go:29] interesting pod/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 21:56:33.128261 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:33.126575    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k" podUID="c5b0cd90-fe64-4446-9a53-9ab9ebaa269e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 21:56:33.456817 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:33.456778    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:33.486914 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:33.486883    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-866b9fd699-2z7wc"]
Oct 30 21:56:33.487914 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:56:33.487884    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f0c5948_fd9a_4d2c_9a59_38d1d2b0bb2f.slice/crio-7237eebf5c317d6a79fe8a8159238eee3420e642e0f6c9fda393c85fa517f17f WatchSource:0}: Error finding container 7237eebf5c317d6a79fe8a8159238eee3420e642e0f6c9fda393c85fa517f17f: Status 404 returned error can't find the container with id 7237eebf5c317d6a79fe8a8159238eee3420e642e0f6c9fda393c85fa517f17f
Oct 30 21:56:34.127443 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:34.127400    4871 patch_prober.go:29] interesting pod/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 21:56:34.127671 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:34.127458    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k" podUID="c5b0cd90-fe64-4446-9a53-9ab9ebaa269e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 21:56:34.129455 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:34.129426    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-866b9fd699-2z7wc" event={"ID":"2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f","Type":"ContainerStarted","Data":"c0e4718851005055e484d1b593da9cea12bfcfa982fcb42c2ea6fe7f720368e6"}
Oct 30 21:56:34.129848 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:34.129485    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-866b9fd699-2z7wc" event={"ID":"2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f","Type":"ContainerStarted","Data":"7237eebf5c317d6a79fe8a8159238eee3420e642e0f6c9fda393c85fa517f17f"}
Oct 30 21:56:34.129848 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:34.129669    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-866b9fd699-2z7wc"
Oct 30 21:56:34.243298 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:34.243223    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-866b9fd699-2z7wc" podStartSLOduration=2.243209226 podStartE2EDuration="2.243209226s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:56:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:56:34.239729138 +0000 UTC m=+1136.417217754" watchObservedRunningTime="2026-10-30 21:56:34.243209226 +0000 UTC m=+1136.420697843"
Oct 30 21:56:34.283903 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:34.283872    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:35.110112 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:35.110061    4871 patch_prober.go:29] interesting pod/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 21:56:35.110316 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:35.110136    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k" podUID="c5b0cd90-fe64-4446-9a53-9ab9ebaa269e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 21:56:35.574617 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:56:35.574571    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:56:36.115085 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:36.115052    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"
Oct 30 21:56:36.115510 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:36.115484    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"
Oct 30 21:56:36.116312 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:36.116291    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"
Oct 30 21:56:37.730921 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:37.730828    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-64f8c8bfbd-l9dfg" podUID="0b56da79-7e87-41b3-9a07-e320389b9833" containerName="payload-processing" containerID="cri-o://c2f1bb98a5a3f41689c3a33ba266bbdf151e2e30f6792901622878a88c03baaf" gracePeriod=25
Oct 30 21:56:38.007816 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:38.007784    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-64f8c8bfbd-l9dfg"
Oct 30 21:56:38.031787 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:38.031750    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:38.043392 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:38.043354    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0b56da79-7e87-41b3-9a07-e320389b9833-kube-api-access-zjq82\" (UniqueName: \"kubernetes.io/projected/0b56da79-7e87-41b3-9a07-e320389b9833-kube-api-access-zjq82\") pod \"0b56da79-7e87-41b3-9a07-e320389b9833\" (UID: \"0b56da79-7e87-41b3-9a07-e320389b9833\") "
Oct 30 21:56:38.043589 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:38.043410    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0b56da79-7e87-41b3-9a07-e320389b9833-provider-credentials\" (UniqueName: \"kubernetes.io/projected/0b56da79-7e87-41b3-9a07-e320389b9833-provider-credentials\") pod \"0b56da79-7e87-41b3-9a07-e320389b9833\" (UID: \"0b56da79-7e87-41b3-9a07-e320389b9833\") "
Oct 30 21:56:38.043589 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:38.043431    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0b56da79-7e87-41b3-9a07-e320389b9833-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b56da79-7e87-41b3-9a07-e320389b9833-plugins-config-volume\") pod \"0b56da79-7e87-41b3-9a07-e320389b9833\" (UID: \"0b56da79-7e87-41b3-9a07-e320389b9833\") "
Oct 30 21:56:38.043589 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:38.043509    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0b56da79-7e87-41b3-9a07-e320389b9833-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/0b56da79-7e87-41b3-9a07-e320389b9833-routing-overlay-volume\") pod \"0b56da79-7e87-41b3-9a07-e320389b9833\" (UID: \"0b56da79-7e87-41b3-9a07-e320389b9833\") "
Oct 30 21:56:38.043950 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:38.043917    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b56da79-7e87-41b3-9a07-e320389b9833-routing-overlay-volume" pod "0b56da79-7e87-41b3-9a07-e320389b9833" (UID: "0b56da79-7e87-41b3-9a07-e320389b9833"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:56:38.044096 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:38.043952    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b56da79-7e87-41b3-9a07-e320389b9833-plugins-config-volume" pod "0b56da79-7e87-41b3-9a07-e320389b9833" (UID: "0b56da79-7e87-41b3-9a07-e320389b9833"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:56:38.044406 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:38.044382    4871 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/0b56da79-7e87-41b3-9a07-e320389b9833-routing-overlay-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:38.044501 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:38.044411    4871 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b56da79-7e87-41b3-9a07-e320389b9833-plugins-config-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:38.046056 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:38.046016    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b56da79-7e87-41b3-9a07-e320389b9833-provider-credentials" pod "0b56da79-7e87-41b3-9a07-e320389b9833" (UID: "0b56da79-7e87-41b3-9a07-e320389b9833"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:38.046187 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:38.046161    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b56da79-7e87-41b3-9a07-e320389b9833-kube-api-access-zjq82" pod "0b56da79-7e87-41b3-9a07-e320389b9833" (UID: "0b56da79-7e87-41b3-9a07-e320389b9833"). InnerVolumeSpecName "kube-api-access-zjq82". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:38.148159 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:38.148120    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjq82\" (UniqueName: \"kubernetes.io/projected/0b56da79-7e87-41b3-9a07-e320389b9833-kube-api-access-zjq82\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:38.148159 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:38.148160    4871 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/0b56da79-7e87-41b3-9a07-e320389b9833-provider-credentials\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:38.164651 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:38.164616    4871 generic.go:356] "Generic (PLEG): container finished" podID="0b56da79-7e87-41b3-9a07-e320389b9833" containerID="c2f1bb98a5a3f41689c3a33ba266bbdf151e2e30f6792901622878a88c03baaf" exitCode=0
Oct 30 21:56:38.164858 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:38.164694    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-64f8c8bfbd-l9dfg"
Oct 30 21:56:38.164858 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:38.164698    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-64f8c8bfbd-l9dfg" event={"ID":"0b56da79-7e87-41b3-9a07-e320389b9833","Type":"ContainerDied","Data":"c2f1bb98a5a3f41689c3a33ba266bbdf151e2e30f6792901622878a88c03baaf"}
Oct 30 21:56:38.164858 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:38.164779    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-64f8c8bfbd-l9dfg" event={"ID":"0b56da79-7e87-41b3-9a07-e320389b9833","Type":"ContainerDied","Data":"6c8c2a8de2f66db2e4854f6a2274c72e4d7769fcf13d3539997bc79b3d6a47ea"}
Oct 30 21:56:38.164858 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:38.164811    4871 scope.go:122] "RemoveContainer" containerID="c2f1bb98a5a3f41689c3a33ba266bbdf151e2e30f6792901622878a88c03baaf"
Oct 30 21:56:38.177795 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:38.177773    4871 scope.go:122] "RemoveContainer" containerID="c2f1bb98a5a3f41689c3a33ba266bbdf151e2e30f6792901622878a88c03baaf"
Oct 30 21:56:38.178187 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:56:38.178165    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2f1bb98a5a3f41689c3a33ba266bbdf151e2e30f6792901622878a88c03baaf\": container with ID starting with c2f1bb98a5a3f41689c3a33ba266bbdf151e2e30f6792901622878a88c03baaf not found: ID does not exist" containerID="c2f1bb98a5a3f41689c3a33ba266bbdf151e2e30f6792901622878a88c03baaf"
Oct 30 21:56:38.178259 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:38.178199    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2f1bb98a5a3f41689c3a33ba266bbdf151e2e30f6792901622878a88c03baaf"} err="failed to get container status \"c2f1bb98a5a3f41689c3a33ba266bbdf151e2e30f6792901622878a88c03baaf\": rpc error: code = NotFound desc = could not find container \"c2f1bb98a5a3f41689c3a33ba266bbdf151e2e30f6792901622878a88c03baaf\": container with ID starting with c2f1bb98a5a3f41689c3a33ba266bbdf151e2e30f6792901622878a88c03baaf not found: ID does not exist"
Oct 30 21:56:38.255271 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:38.255151    4871 scope.go:122] "RemoveContainer" containerID="d1f0c943ec411b33fb7714dbd3ada0416f82107097690f82a6b23bfea589dea5"
Oct 30 21:56:38.265219 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:38.265191    4871 scope.go:122] "RemoveContainer" containerID="9b412d2dcd5731955776dade6ab0a638a312bc13eddb208b78c9e183c38d6af1"
Oct 30 21:56:38.275009 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:38.274987    4871 scope.go:122] "RemoveContainer" containerID="a0fb0f1bd6a507efbd5b58abec574a38ffad9ce1aaa01a9acb8ae7ea75284031"
Oct 30 21:56:38.284012 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:38.283983    4871 scope.go:122] "RemoveContainer" containerID="d1ad1927ef91e1f7f85c3da00f156f8fd73cdfb68a1cbff9f4c71fad02edd90f"
Oct 30 21:56:38.293199 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:38.293179    4871 scope.go:122] "RemoveContainer" containerID="b06975fb9e946bc67a19fd11847216ea377136fefd84ee0c3c7512c7630aed03"
Oct 30 21:56:38.325030 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:38.325002    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-64f8c8bfbd-l9dfg"]
Oct 30 21:56:38.330206 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:38.330177    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-64f8c8bfbd-l9dfg"]
Oct 30 21:56:39.966070 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:39.966033    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0b56da79-7e87-41b3-9a07-e320389b9833" path="/var/lib/kubelet/pods/0b56da79-7e87-41b3-9a07-e320389b9833/volumes"
Oct 30 21:56:43.125261 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:43.125193    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:45.140673 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:45.140631    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-866b9fd699-2z7wc" podUID="2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:56:48.004892 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:48.004835    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:51.964875 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:51.964843    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-59vx9-4jkc6"]
Oct 30 21:56:51.965353 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:51.965339    4871 cpu_manager.go:414] "containerMap: removing container" podUID="0b56da79-7e87-41b3-9a07-e320389b9833" containerName="payload-processing"
Oct 30 21:56:51.965402 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:51.965354    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0b56da79-7e87-41b3-9a07-e320389b9833" containerName="payload-processing"
Oct 30 21:56:51.965457 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:51.965448    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="0b56da79-7e87-41b3-9a07-e320389b9833" containerName="payload-processing"
Oct 30 21:56:51.969108 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:51.969086    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-59vx9-4jkc6"
Oct 30 21:56:52.010183 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:52.010150    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-59vx9-4jkc6"]
Oct 30 21:56:52.053340 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:52.053306    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-4rnkh\""
Oct 30 21:56:52.090035 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:52.089973    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6657b9b-c3d6-441a-acb4-0fd15e194415-trusted-ca-bundle\") pod \"periodic-gathering-59vx9-4jkc6\" (UID: \"f6657b9b-c3d6-441a-acb4-0fd15e194415\") " pod="openshift-insights/periodic-gathering-59vx9-4jkc6"
Oct 30 21:56:52.090197 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:52.090092    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6657b9b-c3d6-441a-acb4-0fd15e194415-service-ca-bundle\") pod \"periodic-gathering-59vx9-4jkc6\" (UID: \"f6657b9b-c3d6-441a-acb4-0fd15e194415\") " pod="openshift-insights/periodic-gathering-59vx9-4jkc6"
Oct 30 21:56:52.090197 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:52.090149    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-779gt\" (UniqueName: \"kubernetes.io/projected/f6657b9b-c3d6-441a-acb4-0fd15e194415-kube-api-access-779gt\") pod \"periodic-gathering-59vx9-4jkc6\" (UID: \"f6657b9b-c3d6-441a-acb4-0fd15e194415\") " pod="openshift-insights/periodic-gathering-59vx9-4jkc6"
Oct 30 21:56:52.090308 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:52.090197    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/f6657b9b-c3d6-441a-acb4-0fd15e194415-archives-path\") pod \"periodic-gathering-59vx9-4jkc6\" (UID: \"f6657b9b-c3d6-441a-acb4-0fd15e194415\") " pod="openshift-insights/periodic-gathering-59vx9-4jkc6"
Oct 30 21:56:52.191970 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:52.191930    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6657b9b-c3d6-441a-acb4-0fd15e194415-trusted-ca-bundle\") pod \"periodic-gathering-59vx9-4jkc6\" (UID: \"f6657b9b-c3d6-441a-acb4-0fd15e194415\") " pod="openshift-insights/periodic-gathering-59vx9-4jkc6"
Oct 30 21:56:52.191970 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:52.191978    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6657b9b-c3d6-441a-acb4-0fd15e194415-service-ca-bundle\") pod \"periodic-gathering-59vx9-4jkc6\" (UID: \"f6657b9b-c3d6-441a-acb4-0fd15e194415\") " pod="openshift-insights/periodic-gathering-59vx9-4jkc6"
Oct 30 21:56:52.192279 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:52.192019    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-779gt\" (UniqueName: \"kubernetes.io/projected/f6657b9b-c3d6-441a-acb4-0fd15e194415-kube-api-access-779gt\") pod \"periodic-gathering-59vx9-4jkc6\" (UID: \"f6657b9b-c3d6-441a-acb4-0fd15e194415\") " pod="openshift-insights/periodic-gathering-59vx9-4jkc6"
Oct 30 21:56:52.192279 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:52.192059    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/f6657b9b-c3d6-441a-acb4-0fd15e194415-archives-path\") pod \"periodic-gathering-59vx9-4jkc6\" (UID: \"f6657b9b-c3d6-441a-acb4-0fd15e194415\") " pod="openshift-insights/periodic-gathering-59vx9-4jkc6"
Oct 30 21:56:52.192510 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:52.192486    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/f6657b9b-c3d6-441a-acb4-0fd15e194415-archives-path\") pod \"periodic-gathering-59vx9-4jkc6\" (UID: \"f6657b9b-c3d6-441a-acb4-0fd15e194415\") " pod="openshift-insights/periodic-gathering-59vx9-4jkc6"
Oct 30 21:56:52.192713 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:52.192690    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6657b9b-c3d6-441a-acb4-0fd15e194415-service-ca-bundle\") pod \"periodic-gathering-59vx9-4jkc6\" (UID: \"f6657b9b-c3d6-441a-acb4-0fd15e194415\") " pod="openshift-insights/periodic-gathering-59vx9-4jkc6"
Oct 30 21:56:52.192820 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:52.192800    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6657b9b-c3d6-441a-acb4-0fd15e194415-trusted-ca-bundle\") pod \"periodic-gathering-59vx9-4jkc6\" (UID: \"f6657b9b-c3d6-441a-acb4-0fd15e194415\") " pod="openshift-insights/periodic-gathering-59vx9-4jkc6"
Oct 30 21:56:52.250658 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:52.250580    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-779gt\" (UniqueName: \"kubernetes.io/projected/f6657b9b-c3d6-441a-acb4-0fd15e194415-kube-api-access-779gt\") pod \"periodic-gathering-59vx9-4jkc6\" (UID: \"f6657b9b-c3d6-441a-acb4-0fd15e194415\") " pod="openshift-insights/periodic-gathering-59vx9-4jkc6"
Oct 30 21:56:52.282635 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:52.282593    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-59vx9-4jkc6"
Oct 30 21:56:52.513345 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:52.513263    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:52.659292 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:52.659260    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-59vx9-4jkc6"]
Oct 30 21:56:52.660474 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:56:52.660449    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6657b9b_c3d6_441a_acb4_0fd15e194415.slice/crio-8a08cb418ddf1ad51dbc38d2fb9484ec1e4daf72795702a0637643513d938182 WatchSource:0}: Error finding container 8a08cb418ddf1ad51dbc38d2fb9484ec1e4daf72795702a0637643513d938182: Status 404 returned error can't find the container with id 8a08cb418ddf1ad51dbc38d2fb9484ec1e4daf72795702a0637643513d938182
Oct 30 21:56:53.321998 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:53.321954    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-59vx9-4jkc6" event={"ID":"f6657b9b-c3d6-441a-acb4-0fd15e194415","Type":"ContainerStarted","Data":"34ced6671113ce1348baf2a457d96f3d862bfb0d0f8529202f69df7c9a2c9eed"}
Oct 30 21:56:53.322504 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:53.322011    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-59vx9-4jkc6" event={"ID":"f6657b9b-c3d6-441a-acb4-0fd15e194415","Type":"ContainerStarted","Data":"8a08cb418ddf1ad51dbc38d2fb9484ec1e4daf72795702a0637643513d938182"}
Oct 30 21:56:53.431859 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:53.431805    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-59vx9-4jkc6" podStartSLOduration=2.431791969 podStartE2EDuration="2.431791969s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:56:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:56:53.431309345 +0000 UTC m=+1155.608797961" watchObservedRunningTime="2026-10-30 21:56:53.431791969 +0000 UTC m=+1155.609280585"
Oct 30 21:56:55.138734 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:55.138692    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-866b9fd699-2z7wc" podUID="2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:56:57.023499 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:56:57.023467    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:00.965117 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:00.965080    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:01.618888 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:01.618855    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:05.138755 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:05.138710    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-866b9fd699-2z7wc" podUID="2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:57:05.568385 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:57:05.568298    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:57:06.218638 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:06.218603    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:10.820418 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:10.820376    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:15.139317 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:15.139276    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-866b9fd699-2z7wc" podUID="2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:57:15.538959 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:15.538875    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:20.043873 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:20.043837    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:23.003374 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:23.003335    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:24.823848 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:24.823799    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:25.138625 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:25.138581    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-866b9fd699-2z7wc" podUID="2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:57:27.072294 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:27.072261    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:29.488700 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:29.488640    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:34.018334 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:34.018303    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:34.620080 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:34.620044    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:35.139001 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:35.138961    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-866b9fd699-2z7wc" podUID="2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:57:35.568944 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:57:35.568852    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:57:38.139638 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:38.139604    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx_8d401654-8a4c-4a79-b2d3-612765312421/extract/0.log"
Oct 30 21:57:38.144327 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:38.144302    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx_8d401654-8a4c-4a79-b2d3-612765312421/extract/0.log"
Oct 30 21:57:38.154424 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:38.154396    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/2.log"
Oct 30 21:57:38.159815 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:38.159781    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/2.log"
Oct 30 21:57:38.163912 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:38.163882    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzntd_2360fddc-52a5-42a5-82ca-a05aff84d217/ovn-acl-logging/0.log"
Oct 30 21:57:38.169361 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:38.169334    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzntd_2360fddc-52a5-42a5-82ca-a05aff84d217/ovn-acl-logging/0.log"
Oct 30 21:57:38.625309 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:38.625192    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:38.800412 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:38.800370    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:42.954380 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:42.954346    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-866b9fd699-2z7wc" podUID="2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:57:43.303363 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:43.303269    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:47.942380 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:47.942327    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:50.931717 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:50.931668    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:51.113403 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:51.113367    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:51.277014 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:51.276922    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:52.760302 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:52.760266    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:52.954855 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:52.954810    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-866b9fd699-2z7wc" podUID="2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:57:57.409789 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:57.409743    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:57.876721 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:57.876685    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-866b9fd699-2z7wc"]
Oct 30 21:57:57.878249 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:57.878199    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-866b9fd699-2z7wc" podUID="2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:57:57.909609 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:57.909569    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-78677bf7-fm4j7"]
Oct 30 21:57:57.914013 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:57.913992    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-78677bf7-fm4j7"
Oct 30 21:57:57.963582 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:57.963550    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-3d0f8113fe57\""
Oct 30 21:57:57.963582 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:57.963551    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-00e4c0ba7157\""
Oct 30 21:57:57.983620 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:57.983587    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-78677bf7-fm4j7"]
Oct 30 21:57:58.088918 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:58.088880    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/33d005ac-b847-4fd5-af8f-5b0162b56610-provider-credentials\") pod \"payload-processing-external-model-78677bf7-fm4j7\" (UID: \"33d005ac-b847-4fd5-af8f-5b0162b56610\") " pod="models-as-a-service/payload-processing-external-model-78677bf7-fm4j7"
Oct 30 21:57:58.089104 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:58.088933    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrxl4\" (UniqueName: \"kubernetes.io/projected/33d005ac-b847-4fd5-af8f-5b0162b56610-kube-api-access-jrxl4\") pod \"payload-processing-external-model-78677bf7-fm4j7\" (UID: \"33d005ac-b847-4fd5-af8f-5b0162b56610\") " pod="models-as-a-service/payload-processing-external-model-78677bf7-fm4j7"
Oct 30 21:57:58.089104 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:58.089069    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33d005ac-b847-4fd5-af8f-5b0162b56610-plugins-config-volume\") pod \"payload-processing-external-model-78677bf7-fm4j7\" (UID: \"33d005ac-b847-4fd5-af8f-5b0162b56610\") " pod="models-as-a-service/payload-processing-external-model-78677bf7-fm4j7"
Oct 30 21:57:58.089228 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:58.089211    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/33d005ac-b847-4fd5-af8f-5b0162b56610-routing-overlay-volume\") pod \"payload-processing-external-model-78677bf7-fm4j7\" (UID: \"33d005ac-b847-4fd5-af8f-5b0162b56610\") " pod="models-as-a-service/payload-processing-external-model-78677bf7-fm4j7"
Oct 30 21:57:58.190420 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:58.190320    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/33d005ac-b847-4fd5-af8f-5b0162b56610-routing-overlay-volume\") pod \"payload-processing-external-model-78677bf7-fm4j7\" (UID: \"33d005ac-b847-4fd5-af8f-5b0162b56610\") " pod="models-as-a-service/payload-processing-external-model-78677bf7-fm4j7"
Oct 30 21:57:58.190420 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:58.190415    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/33d005ac-b847-4fd5-af8f-5b0162b56610-provider-credentials\") pod \"payload-processing-external-model-78677bf7-fm4j7\" (UID: \"33d005ac-b847-4fd5-af8f-5b0162b56610\") " pod="models-as-a-service/payload-processing-external-model-78677bf7-fm4j7"
Oct 30 21:57:58.190652 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:58.190467    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrxl4\" (UniqueName: \"kubernetes.io/projected/33d005ac-b847-4fd5-af8f-5b0162b56610-kube-api-access-jrxl4\") pod \"payload-processing-external-model-78677bf7-fm4j7\" (UID: \"33d005ac-b847-4fd5-af8f-5b0162b56610\") " pod="models-as-a-service/payload-processing-external-model-78677bf7-fm4j7"
Oct 30 21:57:58.190652 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:58.190538    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33d005ac-b847-4fd5-af8f-5b0162b56610-plugins-config-volume\") pod \"payload-processing-external-model-78677bf7-fm4j7\" (UID: \"33d005ac-b847-4fd5-af8f-5b0162b56610\") " pod="models-as-a-service/payload-processing-external-model-78677bf7-fm4j7"
Oct 30 21:57:58.191121 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:58.191092    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/33d005ac-b847-4fd5-af8f-5b0162b56610-routing-overlay-volume\") pod \"payload-processing-external-model-78677bf7-fm4j7\" (UID: \"33d005ac-b847-4fd5-af8f-5b0162b56610\") " pod="models-as-a-service/payload-processing-external-model-78677bf7-fm4j7"
Oct 30 21:57:58.191338 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:58.191317    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33d005ac-b847-4fd5-af8f-5b0162b56610-plugins-config-volume\") pod \"payload-processing-external-model-78677bf7-fm4j7\" (UID: \"33d005ac-b847-4fd5-af8f-5b0162b56610\") " pod="models-as-a-service/payload-processing-external-model-78677bf7-fm4j7"
Oct 30 21:57:58.193207 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:58.193180    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/33d005ac-b847-4fd5-af8f-5b0162b56610-provider-credentials\") pod \"payload-processing-external-model-78677bf7-fm4j7\" (UID: \"33d005ac-b847-4fd5-af8f-5b0162b56610\") " pod="models-as-a-service/payload-processing-external-model-78677bf7-fm4j7"
Oct 30 21:57:58.249428 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:58.249394    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrxl4\" (UniqueName: \"kubernetes.io/projected/33d005ac-b847-4fd5-af8f-5b0162b56610-kube-api-access-jrxl4\") pod \"payload-processing-external-model-78677bf7-fm4j7\" (UID: \"33d005ac-b847-4fd5-af8f-5b0162b56610\") " pod="models-as-a-service/payload-processing-external-model-78677bf7-fm4j7"
Oct 30 21:57:58.528785 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:58.528699    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-78677bf7-fm4j7"
Oct 30 21:57:58.920983 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:58.920942    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-78677bf7-fm4j7"]
Oct 30 21:57:58.921578 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:57:58.921546    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33d005ac_b847_4fd5_af8f_5b0162b56610.slice/crio-c9123c8e980089875a145452c1676f57abed23489b99c13fb5460fb5cbf5861c WatchSource:0}: Error finding container c9123c8e980089875a145452c1676f57abed23489b99c13fb5460fb5cbf5861c: Status 404 returned error can't find the container with id c9123c8e980089875a145452c1676f57abed23489b99c13fb5460fb5cbf5861c
Oct 30 21:57:59.874332 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:59.874288    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-78677bf7-fm4j7" event={"ID":"33d005ac-b847-4fd5-af8f-5b0162b56610","Type":"ContainerStarted","Data":"a6d16e53d08a4a6d60593abc608c1dd94d3331a6d1261e268fcc64e692484c96"}
Oct 30 21:57:59.874737 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:59.874362    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-78677bf7-fm4j7" event={"ID":"33d005ac-b847-4fd5-af8f-5b0162b56610","Type":"ContainerStarted","Data":"c9123c8e980089875a145452c1676f57abed23489b99c13fb5460fb5cbf5861c"}
Oct 30 21:57:59.874737 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:59.874521    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-78677bf7-fm4j7"
Oct 30 21:57:59.988031 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:57:59.987976    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-78677bf7-fm4j7" podStartSLOduration=2.987961747 podStartE2EDuration="2.987961747s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:57:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:57:59.985182074 +0000 UTC m=+1222.162670691" watchObservedRunningTime="2026-10-30 21:57:59.987961747 +0000 UTC m=+1222.165450364"
Oct 30 21:58:00.955258 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:00.955201    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:02.102140 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:02.102106    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:02.877049 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:02.876994    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-866b9fd699-2z7wc" podUID="2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f" containerName="payload-processing" containerID="cri-o://c0e4718851005055e484d1b593da9cea12bfcfa982fcb42c2ea6fe7f720368e6" gracePeriod=25
Oct 30 21:58:03.171023 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:03.170996    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-866b9fd699-2z7wc"
Oct 30 21:58:03.364777 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:03.364033    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f-provider-credentials\" (UniqueName: \"kubernetes.io/projected/2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f-provider-credentials\") pod \"2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f\" (UID: \"2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f\") "
Oct 30 21:58:03.364777 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:03.364148    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f-kube-api-access-mnbgx\" (UniqueName: \"kubernetes.io/projected/2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f-kube-api-access-mnbgx\") pod \"2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f\" (UID: \"2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f\") "
Oct 30 21:58:03.364777 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:03.364171    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f-plugins-config-volume\") pod \"2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f\" (UID: \"2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f\") "
Oct 30 21:58:03.364777 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:03.364311    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f-routing-overlay-volume\") pod \"2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f\" (UID: \"2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f\") "
Oct 30 21:58:03.365146 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:03.364887    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f-plugins-config-volume" pod "2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f" (UID: "2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:58:03.365146 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:03.364920    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f-routing-overlay-volume" pod "2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f" (UID: "2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:58:03.367621 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:03.367585    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f-kube-api-access-mnbgx" pod "2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f" (UID: "2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f"). InnerVolumeSpecName "kube-api-access-mnbgx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:58:03.368080 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:03.368051    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f-provider-credentials" pod "2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f" (UID: "2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:58:03.465987 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:03.465903    4871 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f-provider-credentials\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:58:03.465987 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:03.465935    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mnbgx\" (UniqueName: \"kubernetes.io/projected/2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f-kube-api-access-mnbgx\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:58:03.465987 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:03.465946    4871 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f-plugins-config-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:58:03.465987 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:03.465956    4871 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f-routing-overlay-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:58:03.907900 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:03.907861    4871 generic.go:356] "Generic (PLEG): container finished" podID="2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f" containerID="c0e4718851005055e484d1b593da9cea12bfcfa982fcb42c2ea6fe7f720368e6" exitCode=0
Oct 30 21:58:03.908106 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:03.907939    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-866b9fd699-2z7wc"
Oct 30 21:58:03.908106 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:03.907956    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-866b9fd699-2z7wc" event={"ID":"2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f","Type":"ContainerDied","Data":"c0e4718851005055e484d1b593da9cea12bfcfa982fcb42c2ea6fe7f720368e6"}
Oct 30 21:58:03.908106 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:03.908065    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-866b9fd699-2z7wc" event={"ID":"2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f","Type":"ContainerDied","Data":"7237eebf5c317d6a79fe8a8159238eee3420e642e0f6c9fda393c85fa517f17f"}
Oct 30 21:58:03.908106 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:03.908097    4871 scope.go:122] "RemoveContainer" containerID="c0e4718851005055e484d1b593da9cea12bfcfa982fcb42c2ea6fe7f720368e6"
Oct 30 21:58:03.919121 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:03.919102    4871 scope.go:122] "RemoveContainer" containerID="c0e4718851005055e484d1b593da9cea12bfcfa982fcb42c2ea6fe7f720368e6"
Oct 30 21:58:03.919436 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:58:03.919417    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0e4718851005055e484d1b593da9cea12bfcfa982fcb42c2ea6fe7f720368e6\": container with ID starting with c0e4718851005055e484d1b593da9cea12bfcfa982fcb42c2ea6fe7f720368e6 not found: ID does not exist" containerID="c0e4718851005055e484d1b593da9cea12bfcfa982fcb42c2ea6fe7f720368e6"
Oct 30 21:58:03.919502 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:03.919444    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0e4718851005055e484d1b593da9cea12bfcfa982fcb42c2ea6fe7f720368e6"} err="failed to get container status \"c0e4718851005055e484d1b593da9cea12bfcfa982fcb42c2ea6fe7f720368e6\": rpc error: code = NotFound desc = could not find container \"c0e4718851005055e484d1b593da9cea12bfcfa982fcb42c2ea6fe7f720368e6\": container with ID starting with c0e4718851005055e484d1b593da9cea12bfcfa982fcb42c2ea6fe7f720368e6 not found: ID does not exist"
Oct 30 21:58:04.111333 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:04.111286    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-866b9fd699-2z7wc"]
Oct 30 21:58:04.148302 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:04.148263    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-866b9fd699-2z7wc"]
Oct 30 21:58:05.692574 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:58:05.692536    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:58:05.981261 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:05.980744    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f" path="/var/lib/kubelet/pods/2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f/volumes"
Oct 30 21:58:07.436204 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:07.436160    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:10.888135 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:10.888101    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="models-as-a-service/payload-processing-external-model-78677bf7-fm4j7"
Oct 30 21:58:12.205369 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:12.205324    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:16.830308 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:16.830269    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:21.436020 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:21.435978    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:26.904208 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:26.904150    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:28.800349 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:28.800307    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"]
Oct 30 21:58:28.816259 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:28.805173    4871 cpu_manager.go:414] "containerMap: removing container" podUID="2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f" containerName="payload-processing"
Oct 30 21:58:28.820360 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:28.819340    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f" containerName="payload-processing"
Oct 30 21:58:28.820360 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:28.819595    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="2f0c5948-fd9a-4d2c-9a59-38d1d2b0bb2f" containerName="payload-processing"
Oct 30 21:58:28.823993 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:28.823971    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"
Oct 30 21:58:28.899805 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:28.899765    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33920f19-5306-4275-b275-3433be348592-credential-socket\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql\" (UID: \"33920f19-5306-4275-b275-3433be348592\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"
Oct 30 21:58:28.900075 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:28.900034    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33920f19-5306-4275-b275-3433be348592-istio-podinfo\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql\" (UID: \"33920f19-5306-4275-b275-3433be348592\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"
Oct 30 21:58:28.900342 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:28.900227    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33920f19-5306-4275-b275-3433be348592-istio-data\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql\" (UID: \"33920f19-5306-4275-b275-3433be348592\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"
Oct 30 21:58:28.900625 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:28.900518    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33920f19-5306-4275-b275-3433be348592-istio-token\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql\" (UID: \"33920f19-5306-4275-b275-3433be348592\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"
Oct 30 21:58:28.900625 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:28.900573    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33920f19-5306-4275-b275-3433be348592-workload-socket\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql\" (UID: \"33920f19-5306-4275-b275-3433be348592\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"
Oct 30 21:58:28.900868 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:28.900834    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33920f19-5306-4275-b275-3433be348592-istiod-ca-cert\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql\" (UID: \"33920f19-5306-4275-b275-3433be348592\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"
Oct 30 21:58:28.901132 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:28.901065    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssq4g\" (UniqueName: \"kubernetes.io/projected/33920f19-5306-4275-b275-3433be348592-kube-api-access-ssq4g\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql\" (UID: \"33920f19-5306-4275-b275-3433be348592\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"
Oct 30 21:58:28.920518 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:28.920445    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33920f19-5306-4275-b275-3433be348592-workload-certs\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql\" (UID: \"33920f19-5306-4275-b275-3433be348592\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"
Oct 30 21:58:28.920790 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:28.920739    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33920f19-5306-4275-b275-3433be348592-istio-envoy\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql\" (UID: \"33920f19-5306-4275-b275-3433be348592\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"
Oct 30 21:58:29.094205 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:29.053186    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33920f19-5306-4275-b275-3433be348592-istio-token\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql\" (UID: \"33920f19-5306-4275-b275-3433be348592\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"
Oct 30 21:58:29.094205 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:29.053255    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33920f19-5306-4275-b275-3433be348592-workload-socket\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql\" (UID: \"33920f19-5306-4275-b275-3433be348592\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"
Oct 30 21:58:29.094205 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:29.053317    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33920f19-5306-4275-b275-3433be348592-istiod-ca-cert\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql\" (UID: \"33920f19-5306-4275-b275-3433be348592\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"
Oct 30 21:58:29.094205 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:29.053385    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssq4g\" (UniqueName: \"kubernetes.io/projected/33920f19-5306-4275-b275-3433be348592-kube-api-access-ssq4g\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql\" (UID: \"33920f19-5306-4275-b275-3433be348592\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"
Oct 30 21:58:29.094205 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:29.053454    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33920f19-5306-4275-b275-3433be348592-workload-certs\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql\" (UID: \"33920f19-5306-4275-b275-3433be348592\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"
Oct 30 21:58:29.094205 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:29.053493    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33920f19-5306-4275-b275-3433be348592-istio-envoy\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql\" (UID: \"33920f19-5306-4275-b275-3433be348592\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"
Oct 30 21:58:29.094205 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:29.053593    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33920f19-5306-4275-b275-3433be348592-credential-socket\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql\" (UID: \"33920f19-5306-4275-b275-3433be348592\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"
Oct 30 21:58:29.094205 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:29.053621    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33920f19-5306-4275-b275-3433be348592-istio-podinfo\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql\" (UID: \"33920f19-5306-4275-b275-3433be348592\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"
Oct 30 21:58:29.094205 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:29.053650    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33920f19-5306-4275-b275-3433be348592-istio-data\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql\" (UID: \"33920f19-5306-4275-b275-3433be348592\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"
Oct 30 21:58:29.094205 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:29.054117    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33920f19-5306-4275-b275-3433be348592-istio-data\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql\" (UID: \"33920f19-5306-4275-b275-3433be348592\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"
Oct 30 21:58:29.094205 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:29.054711    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33920f19-5306-4275-b275-3433be348592-workload-socket\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql\" (UID: \"33920f19-5306-4275-b275-3433be348592\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"
Oct 30 21:58:29.094205 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:29.055384    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33920f19-5306-4275-b275-3433be348592-istiod-ca-cert\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql\" (UID: \"33920f19-5306-4275-b275-3433be348592\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"
Oct 30 21:58:29.094205 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:29.055842    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33920f19-5306-4275-b275-3433be348592-workload-certs\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql\" (UID: \"33920f19-5306-4275-b275-3433be348592\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"
Oct 30 21:58:29.094205 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:29.093686    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33920f19-5306-4275-b275-3433be348592-credential-socket\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql\" (UID: \"33920f19-5306-4275-b275-3433be348592\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"
Oct 30 21:58:29.130907 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:29.121663    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:29.229178 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:29.195930    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"]
Oct 30 21:58:29.259788 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:29.233925    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33920f19-5306-4275-b275-3433be348592-istio-token\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql\" (UID: \"33920f19-5306-4275-b275-3433be348592\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"
Oct 30 21:58:29.293432 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:29.269872    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33920f19-5306-4275-b275-3433be348592-istio-podinfo\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql\" (UID: \"33920f19-5306-4275-b275-3433be348592\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"
Oct 30 21:58:29.341716 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:29.320065    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssq4g\" (UniqueName: \"kubernetes.io/projected/33920f19-5306-4275-b275-3433be348592-kube-api-access-ssq4g\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql\" (UID: \"33920f19-5306-4275-b275-3433be348592\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"
Oct 30 21:58:29.341716 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:29.341640    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33920f19-5306-4275-b275-3433be348592-istio-envoy\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql\" (UID: \"33920f19-5306-4275-b275-3433be348592\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"
Oct 30 21:58:29.507150 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:29.507075    4871 generic.go:356] "Generic (PLEG): container finished" podID="f6657b9b-c3d6-441a-acb4-0fd15e194415" containerID="34ced6671113ce1348baf2a457d96f3d862bfb0d0f8529202f69df7c9a2c9eed" exitCode=0
Oct 30 21:58:29.525370 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:29.508996    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-59vx9-4jkc6" event={"ID":"f6657b9b-c3d6-441a-acb4-0fd15e194415","Type":"ContainerDied","Data":"34ced6671113ce1348baf2a457d96f3d862bfb0d0f8529202f69df7c9a2c9eed"}
Oct 30 21:58:29.526788 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:29.526464    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"
Oct 30 21:58:30.048901 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:30.048871    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"]
Oct 30 21:58:30.050299 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:58:30.050271    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33920f19_5306_4275_b275_3433be348592.slice/crio-4878d867837ff98711b5f3b2763678d42cbf569da6daaf889305d081d467cf16 WatchSource:0}: Error finding container 4878d867837ff98711b5f3b2763678d42cbf569da6daaf889305d081d467cf16: Status 404 returned error can't find the container with id 4878d867837ff98711b5f3b2763678d42cbf569da6daaf889305d081d467cf16
Oct 30 21:58:30.053189 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:30.053151    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:58:30.053301 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:30.053262    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:58:30.053348 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:30.053306    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:58:30.536376 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:30.536279    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql" event={"ID":"33920f19-5306-4275-b275-3433be348592","Type":"ContainerStarted","Data":"903b49276da1b65dc699be10ae7773ed65cba1c379157ea01016890a10e3fb8b"}
Oct 30 21:58:30.536376 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:30.536358    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql" event={"ID":"33920f19-5306-4275-b275-3433be348592","Type":"ContainerStarted","Data":"4878d867837ff98711b5f3b2763678d42cbf569da6daaf889305d081d467cf16"}
Oct 30 21:58:30.653224 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:30.653172    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql" podStartSLOduration=2.653154982 podStartE2EDuration="2.653154982s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:58:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:58:30.650163608 +0000 UTC m=+1252.827652224" watchObservedRunningTime="2026-10-30 21:58:30.653154982 +0000 UTC m=+1252.830643600"
Oct 30 21:58:30.729943 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:30.729905    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-59vx9-4jkc6"
Oct 30 21:58:30.850862 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:30.850829    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f6657b9b-c3d6-441a-acb4-0fd15e194415-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6657b9b-c3d6-441a-acb4-0fd15e194415-service-ca-bundle\") pod \"f6657b9b-c3d6-441a-acb4-0fd15e194415\" (UID: \"f6657b9b-c3d6-441a-acb4-0fd15e194415\") "
Oct 30 21:58:30.850862 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:30.850865    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f6657b9b-c3d6-441a-acb4-0fd15e194415-kube-api-access-779gt\" (UniqueName: \"kubernetes.io/projected/f6657b9b-c3d6-441a-acb4-0fd15e194415-kube-api-access-779gt\") pod \"f6657b9b-c3d6-441a-acb4-0fd15e194415\" (UID: \"f6657b9b-c3d6-441a-acb4-0fd15e194415\") "
Oct 30 21:58:30.851078 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:30.850908    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f6657b9b-c3d6-441a-acb4-0fd15e194415-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6657b9b-c3d6-441a-acb4-0fd15e194415-trusted-ca-bundle\") pod \"f6657b9b-c3d6-441a-acb4-0fd15e194415\" (UID: \"f6657b9b-c3d6-441a-acb4-0fd15e194415\") "
Oct 30 21:58:30.851078 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:30.850980    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6657b9b-c3d6-441a-acb4-0fd15e194415-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/f6657b9b-c3d6-441a-acb4-0fd15e194415-archives-path\") pod \"f6657b9b-c3d6-441a-acb4-0fd15e194415\" (UID: \"f6657b9b-c3d6-441a-acb4-0fd15e194415\") "
Oct 30 21:58:30.851390 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:30.851356    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6657b9b-c3d6-441a-acb4-0fd15e194415-service-ca-bundle" pod "f6657b9b-c3d6-441a-acb4-0fd15e194415" (UID: "f6657b9b-c3d6-441a-acb4-0fd15e194415"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:58:30.851390 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:30.851362    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6657b9b-c3d6-441a-acb4-0fd15e194415-archives-path" pod "f6657b9b-c3d6-441a-acb4-0fd15e194415" (UID: "f6657b9b-c3d6-441a-acb4-0fd15e194415"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:58:30.851528 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:30.851410    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6657b9b-c3d6-441a-acb4-0fd15e194415-trusted-ca-bundle" pod "f6657b9b-c3d6-441a-acb4-0fd15e194415" (UID: "f6657b9b-c3d6-441a-acb4-0fd15e194415"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:58:30.851528 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:30.851451    4871 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6657b9b-c3d6-441a-acb4-0fd15e194415-service-ca-bundle\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:58:30.851528 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:30.851464    4871 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/f6657b9b-c3d6-441a-acb4-0fd15e194415-archives-path\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:58:30.853811 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:30.853789    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6657b9b-c3d6-441a-acb4-0fd15e194415-kube-api-access-779gt" pod "f6657b9b-c3d6-441a-acb4-0fd15e194415" (UID: "f6657b9b-c3d6-441a-acb4-0fd15e194415"). InnerVolumeSpecName "kube-api-access-779gt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:58:30.952128 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:30.952082    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-779gt\" (UniqueName: \"kubernetes.io/projected/f6657b9b-c3d6-441a-acb4-0fd15e194415-kube-api-access-779gt\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:58:30.952128 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:30.952119    4871 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6657b9b-c3d6-441a-acb4-0fd15e194415-trusted-ca-bundle\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:58:31.531403 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:31.531304    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"
Oct 30 21:58:31.533457 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:31.533417    4871 patch_prober.go:29] interesting pod/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Oct 30 21:58:31.533628 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:31.533490    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql" podUID="33920f19-5306-4275-b275-3433be348592" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Oct 30 21:58:31.546308 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:31.546274    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-59vx9-4jkc6"
Oct 30 21:58:31.546487 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:31.546300    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-59vx9-4jkc6" event={"ID":"f6657b9b-c3d6-441a-acb4-0fd15e194415","Type":"ContainerDied","Data":"8a08cb418ddf1ad51dbc38d2fb9484ec1e4daf72795702a0637643513d938182"}
Oct 30 21:58:31.546487 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:31.546358    4871 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8a08cb418ddf1ad51dbc38d2fb9484ec1e4daf72795702a0637643513d938182"
Oct 30 21:58:31.941109 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:31.941062    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:32.529702 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:32.529663    4871 patch_prober.go:29] interesting pod/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Oct 30 21:58:32.529909 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:32.529741    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql" podUID="33920f19-5306-4275-b275-3433be348592" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Oct 30 21:58:32.853192 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:32.853064    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:33.026835 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:33.026797    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:33.531263 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:33.527656    4871 patch_prober.go:29] interesting pod/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Oct 30 21:58:33.531263 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:33.527725    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql" podUID="33920f19-5306-4275-b275-3433be348592" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Oct 30 21:58:34.529261 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:34.528704    4871 patch_prober.go:29] interesting pod/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Oct 30 21:58:34.529261 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:34.528793    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql" podUID="33920f19-5306-4275-b275-3433be348592" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Oct 30 21:58:34.967706 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:34.967665    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:35.531262 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:35.527668    4871 patch_prober.go:29] interesting pod/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Oct 30 21:58:35.531262 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:35.527742    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql" podUID="33920f19-5306-4275-b275-3433be348592" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Oct 30 21:58:35.570318 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:58:35.570276    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:58:36.528014 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:36.527962    4871 patch_prober.go:29] interesting pod/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Oct 30 21:58:36.528206 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:36.528044    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql" podUID="33920f19-5306-4275-b275-3433be348592" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Oct 30 21:58:36.702277 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:36.702219    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:37.531822 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:37.531789    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"
Oct 30 21:58:37.532302 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:37.532270    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"
Oct 30 21:58:37.533052 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:37.533033    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"
Oct 30 21:58:38.932294 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:38.931692    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"]
Oct 30 21:58:38.932804 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:38.932550    4871 cpu_manager.go:414] "containerMap: removing container" podUID="f6657b9b-c3d6-441a-acb4-0fd15e194415" containerName="insights-gathering"
Oct 30 21:58:38.932804 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:38.932570    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f6657b9b-c3d6-441a-acb4-0fd15e194415" containerName="insights-gathering"
Oct 30 21:58:38.932804 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:38.932725    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="f6657b9b-c3d6-441a-acb4-0fd15e194415" containerName="insights-gathering"
Oct 30 21:58:38.943417 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:38.943383    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"
Oct 30 21:58:39.017727 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:39.017691    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"]
Oct 30 21:58:39.082347 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:39.082307    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmlvj\" (UniqueName: \"kubernetes.io/projected/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-kube-api-access-wmlvj\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs\" (UID: \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"
Oct 30 21:58:39.082347 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:39.082353    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-credential-socket\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs\" (UID: \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"
Oct 30 21:58:39.082633 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:39.082372    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-workload-certs\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs\" (UID: \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"
Oct 30 21:58:39.082633 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:39.082486    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-workload-socket\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs\" (UID: \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"
Oct 30 21:58:39.082741 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:39.082661    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-istio-podinfo\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs\" (UID: \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"
Oct 30 21:58:39.082741 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:39.082730    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-istiod-ca-cert\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs\" (UID: \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"
Oct 30 21:58:39.082849 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:39.082794    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-istio-envoy\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs\" (UID: \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"
Oct 30 21:58:39.082849 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:39.082821    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-istio-data\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs\" (UID: \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"
Oct 30 21:58:39.082955 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:39.082937    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-istio-token\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs\" (UID: \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"
Oct 30 21:58:39.184008 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:39.183910    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmlvj\" (UniqueName: \"kubernetes.io/projected/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-kube-api-access-wmlvj\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs\" (UID: \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"
Oct 30 21:58:39.184008 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:39.183954    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-credential-socket\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs\" (UID: \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"
Oct 30 21:58:39.184008 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:39.183985    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-workload-certs\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs\" (UID: \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"
Oct 30 21:58:39.184461 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:39.184033    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-workload-socket\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs\" (UID: \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"
Oct 30 21:58:39.184461 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:39.184167    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-istio-podinfo\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs\" (UID: \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"
Oct 30 21:58:39.184461 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:39.184265    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-istiod-ca-cert\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs\" (UID: \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"
Oct 30 21:58:39.184461 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:39.184311    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-istio-envoy\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs\" (UID: \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"
Oct 30 21:58:39.184461 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:39.184338    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-istio-data\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs\" (UID: \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"
Oct 30 21:58:39.184461 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:39.184410    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-istio-token\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs\" (UID: \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"
Oct 30 21:58:39.184806 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:39.184522    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-workload-socket\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs\" (UID: \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"
Oct 30 21:58:39.184806 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:39.184693    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-istio-data\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs\" (UID: \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"
Oct 30 21:58:39.184806 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:39.184757    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-credential-socket\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs\" (UID: \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"
Oct 30 21:58:39.184984 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:39.184920    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-workload-certs\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs\" (UID: \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"
Oct 30 21:58:39.185414 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:39.185363    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-istiod-ca-cert\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs\" (UID: \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"
Oct 30 21:58:39.187051 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:39.187026    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-istio-envoy\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs\" (UID: \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"
Oct 30 21:58:39.187542 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:39.187518    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-istio-podinfo\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs\" (UID: \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"
Oct 30 21:58:39.242164 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:39.242128    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-istio-token\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs\" (UID: \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"
Oct 30 21:58:39.242316 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:39.242283    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmlvj\" (UniqueName: \"kubernetes.io/projected/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-kube-api-access-wmlvj\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs\" (UID: \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"
Oct 30 21:58:39.275483 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:39.275447    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"
Oct 30 21:58:40.064280 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:40.058261    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:58:40.064280 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:40.058424    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:58:40.064280 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:40.058520    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:58:40.121393 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:40.105048    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"]
Oct 30 21:58:40.204789 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:40.204745    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"]
Oct 30 21:58:40.298382 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:40.296744    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"
Oct 30 21:58:40.298382 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:40.297813    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"]
Oct 30 21:58:40.396998 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:40.396571    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ce15ea8-02f4-4b1a-8670-937679cd7e57-workload-socket\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx\" (UID: \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"
Oct 30 21:58:40.396998 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:40.396624    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ce15ea8-02f4-4b1a-8670-937679cd7e57-istio-podinfo\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx\" (UID: \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"
Oct 30 21:58:40.396998 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:40.396665    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h68z5\" (UniqueName: \"kubernetes.io/projected/5ce15ea8-02f4-4b1a-8670-937679cd7e57-kube-api-access-h68z5\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx\" (UID: \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"
Oct 30 21:58:40.396998 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:40.396711    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ce15ea8-02f4-4b1a-8670-937679cd7e57-workload-certs\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx\" (UID: \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"
Oct 30 21:58:40.396998 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:40.396737    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ce15ea8-02f4-4b1a-8670-937679cd7e57-istio-envoy\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx\" (UID: \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"
Oct 30 21:58:40.396998 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:40.396823    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ce15ea8-02f4-4b1a-8670-937679cd7e57-istio-data\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx\" (UID: \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"
Oct 30 21:58:40.396998 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:40.396856    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ce15ea8-02f4-4b1a-8670-937679cd7e57-istiod-ca-cert\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx\" (UID: \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"
Oct 30 21:58:40.396998 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:40.396915    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ce15ea8-02f4-4b1a-8670-937679cd7e57-credential-socket\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx\" (UID: \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"
Oct 30 21:58:40.397566 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:40.397009    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ce15ea8-02f4-4b1a-8670-937679cd7e57-istio-token\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx\" (UID: \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"
Oct 30 21:58:40.499481 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:40.497984    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ce15ea8-02f4-4b1a-8670-937679cd7e57-istio-data\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx\" (UID: \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"
Oct 30 21:58:40.499481 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:40.498080    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ce15ea8-02f4-4b1a-8670-937679cd7e57-istiod-ca-cert\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx\" (UID: \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"
Oct 30 21:58:40.499481 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:40.498131    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ce15ea8-02f4-4b1a-8670-937679cd7e57-credential-socket\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx\" (UID: \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"
Oct 30 21:58:40.499481 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:40.498209    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ce15ea8-02f4-4b1a-8670-937679cd7e57-istio-token\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx\" (UID: \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"
Oct 30 21:58:40.499481 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:40.498317    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ce15ea8-02f4-4b1a-8670-937679cd7e57-workload-socket\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx\" (UID: \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"
Oct 30 21:58:40.499481 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:40.498348    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ce15ea8-02f4-4b1a-8670-937679cd7e57-istio-podinfo\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx\" (UID: \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"
Oct 30 21:58:40.499481 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:40.498379    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h68z5\" (UniqueName: \"kubernetes.io/projected/5ce15ea8-02f4-4b1a-8670-937679cd7e57-kube-api-access-h68z5\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx\" (UID: \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"
Oct 30 21:58:40.499481 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:40.498419    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ce15ea8-02f4-4b1a-8670-937679cd7e57-workload-certs\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx\" (UID: \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"
Oct 30 21:58:40.499481 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:40.498739    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ce15ea8-02f4-4b1a-8670-937679cd7e57-istio-data\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx\" (UID: \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"
Oct 30 21:58:40.499481 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:40.498791    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ce15ea8-02f4-4b1a-8670-937679cd7e57-istio-envoy\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx\" (UID: \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"
Oct 30 21:58:40.499481 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:40.498825    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ce15ea8-02f4-4b1a-8670-937679cd7e57-workload-certs\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx\" (UID: \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"
Oct 30 21:58:40.499481 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:40.499213    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ce15ea8-02f4-4b1a-8670-937679cd7e57-credential-socket\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx\" (UID: \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"
Oct 30 21:58:40.499481 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:40.499412    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ce15ea8-02f4-4b1a-8670-937679cd7e57-workload-socket\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx\" (UID: \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"
Oct 30 21:58:40.499481 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:40.499478    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ce15ea8-02f4-4b1a-8670-937679cd7e57-istiod-ca-cert\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx\" (UID: \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"
Oct 30 21:58:40.501959 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:40.501935    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ce15ea8-02f4-4b1a-8670-937679cd7e57-istio-envoy\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx\" (UID: \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"
Oct 30 21:58:40.502208 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:40.502188    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ce15ea8-02f4-4b1a-8670-937679cd7e57-istio-podinfo\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx\" (UID: \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"
Oct 30 21:58:40.593601 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:40.592853    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ce15ea8-02f4-4b1a-8670-937679cd7e57-istio-token\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx\" (UID: \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"
Oct 30 21:58:40.593601 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:40.593496    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h68z5\" (UniqueName: \"kubernetes.io/projected/5ce15ea8-02f4-4b1a-8670-937679cd7e57-kube-api-access-h68z5\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx\" (UID: \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"
Oct 30 21:58:40.699638 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:40.699407    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"
Oct 30 21:58:40.711504 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:40.711462    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs" event={"ID":"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd","Type":"ContainerStarted","Data":"b470061233344cef6995cfb7ed13d4b9b8096dfba147259a4963308212df1ca3"}
Oct 30 21:58:40.711672 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:40.711537    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs" event={"ID":"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd","Type":"ContainerStarted","Data":"9e83ba591d7bfae9edad8e9231b2b1cb1b29303921780664380bb2c18759bdc7"}
Oct 30 21:58:40.849539 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:40.848140    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs" podStartSLOduration=2.8481209610000002 podStartE2EDuration="2.848120961s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:58:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:58:40.847484945 +0000 UTC m=+1263.024973577" watchObservedRunningTime="2026-10-30 21:58:40.848120961 +0000 UTC m=+1263.025609579"
Oct 30 21:58:41.176072 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:41.176039    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"]
Oct 30 21:58:41.188752 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:41.188692    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:58:41.188897 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:41.188793    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:58:41.188897 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:41.188838    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:58:41.276301 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:41.276272    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"
Oct 30 21:58:41.277980 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:41.277954    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused" start-of-body=
Oct 30 21:58:41.278103 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:41.278001    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs" podUID="78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused"
Oct 30 21:58:41.631007 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:41.630876    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:41.747910 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:41.747778    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx" event={"ID":"5ce15ea8-02f4-4b1a-8670-937679cd7e57","Type":"ContainerStarted","Data":"4a8517c00ea94add2fac3695fcad02e46e56ca62c0e957b7f6050c5a49b0e9ed"}
Oct 30 21:58:41.748336 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:41.748311    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx" event={"ID":"5ce15ea8-02f4-4b1a-8670-937679cd7e57","Type":"ContainerStarted","Data":"834e37586623ab61db68805d51e8a86df9eecc4f74ec739a2388de6a48bf4d6b"}
Oct 30 21:58:41.870510 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:41.870447    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx" podStartSLOduration=1.87043383 podStartE2EDuration="1.87043383s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:58:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:58:41.866984072 +0000 UTC m=+1264.044472690" watchObservedRunningTime="2026-10-30 21:58:41.87043383 +0000 UTC m=+1264.047922446"
Oct 30 21:58:42.276656 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:42.276613    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused" start-of-body=
Oct 30 21:58:42.277231 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:42.276680    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs" podUID="78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused"
Oct 30 21:58:42.700709 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:42.700679    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"
Oct 30 21:58:42.701937 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:42.701910    4871 patch_prober.go:29] interesting pod/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 21:58:42.702037 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:42.701963    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx" podUID="5ce15ea8-02f4-4b1a-8670-937679cd7e57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 21:58:43.276624 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:43.276584    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused" start-of-body=
Oct 30 21:58:43.276837 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:43.276649    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs" podUID="78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused"
Oct 30 21:58:43.496703 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:43.496670    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:43.700372 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:43.700338    4871 patch_prober.go:29] interesting pod/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 21:58:43.700561 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:43.700390    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx" podUID="5ce15ea8-02f4-4b1a-8670-937679cd7e57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 21:58:44.276879 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:44.276823    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused" start-of-body=
Oct 30 21:58:44.277407 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:44.276894    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs" podUID="78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused"
Oct 30 21:58:44.360320 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:44.348211    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:44.713270 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:44.712383    4871 patch_prober.go:29] interesting pod/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 21:58:44.713270 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:44.712447    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx" podUID="5ce15ea8-02f4-4b1a-8670-937679cd7e57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 21:58:45.276130 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:45.276098    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused" start-of-body=
Oct 30 21:58:45.276362 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:45.276317    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs" podUID="78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused"
Oct 30 21:58:45.714264 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:45.702849    4871 patch_prober.go:29] interesting pod/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 21:58:45.714264 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:45.702913    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx" podUID="5ce15ea8-02f4-4b1a-8670-937679cd7e57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 21:58:46.276871 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:46.276349    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused" start-of-body=
Oct 30 21:58:46.276871 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:46.276416    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs" podUID="78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused"
Oct 30 21:58:46.667902 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:46.667861    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:46.710264 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:46.700490    4871 patch_prober.go:29] interesting pod/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 21:58:46.710264 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:46.700560    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx" podUID="5ce15ea8-02f4-4b1a-8670-937679cd7e57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 21:58:47.276541 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:47.276501    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused" start-of-body=
Oct 30 21:58:47.276936 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:47.276556    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs" podUID="78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused"
Oct 30 21:58:47.700390 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:47.700335    4871 patch_prober.go:29] interesting pod/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 21:58:47.700567 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:47.700418    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx" podUID="5ce15ea8-02f4-4b1a-8670-937679cd7e57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 21:58:48.377432 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:48.377401    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"
Oct 30 21:58:48.378037 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:48.378011    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"
Oct 30 21:58:48.380491 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:48.380461    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"
Oct 30 21:58:49.182174 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:49.182128    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"
Oct 30 21:58:49.183370 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:49.183339    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"
Oct 30 21:58:49.184211 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:49.184191    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"
Oct 30 21:58:51.309653 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:51.309616    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:56.001719 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:56.001686    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:59.120327 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:59.120284    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3bdc7244-nsmqj"]
Oct 30 21:58:59.125061 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:59.125034    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3bdc7244-nsmqj"
Oct 30 21:58:59.174907 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:59.174873    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:58:59.197777 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:59.197738    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cc269490-09f2-4b2b-ba6d-df674223d731-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3bdc7244-nsmqj\" (UID: \"cc269490-09f2-4b2b-ba6d-df674223d731\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3bdc7244-nsmqj"
Oct 30 21:58:59.197777 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:59.197782    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cc269490-09f2-4b2b-ba6d-df674223d731-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3bdc7244-nsmqj\" (UID: \"cc269490-09f2-4b2b-ba6d-df674223d731\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3bdc7244-nsmqj"
Oct 30 21:58:59.198011 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:59.197904    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2sfl\" (UniqueName: \"kubernetes.io/projected/cc269490-09f2-4b2b-ba6d-df674223d731-kube-api-access-g2sfl\") pod \"maas-api-revoke-sub-3bdc7244-nsmqj\" (UID: \"cc269490-09f2-4b2b-ba6d-df674223d731\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3bdc7244-nsmqj"
Oct 30 21:58:59.204448 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:59.204407    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mlvtf\""
Oct 30 21:58:59.226125 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:59.226094    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3bdc7244-nsmqj"]
Oct 30 21:58:59.298842 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:59.298806    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cc269490-09f2-4b2b-ba6d-df674223d731-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3bdc7244-nsmqj\" (UID: \"cc269490-09f2-4b2b-ba6d-df674223d731\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3bdc7244-nsmqj"
Oct 30 21:58:59.298842 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:59.298844    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cc269490-09f2-4b2b-ba6d-df674223d731-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3bdc7244-nsmqj\" (UID: \"cc269490-09f2-4b2b-ba6d-df674223d731\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3bdc7244-nsmqj"
Oct 30 21:58:59.299116 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:59.298982    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2sfl\" (UniqueName: \"kubernetes.io/projected/cc269490-09f2-4b2b-ba6d-df674223d731-kube-api-access-g2sfl\") pod \"maas-api-revoke-sub-3bdc7244-nsmqj\" (UID: \"cc269490-09f2-4b2b-ba6d-df674223d731\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3bdc7244-nsmqj"
Oct 30 21:58:59.299645 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:59.299618    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cc269490-09f2-4b2b-ba6d-df674223d731-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3bdc7244-nsmqj\" (UID: \"cc269490-09f2-4b2b-ba6d-df674223d731\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3bdc7244-nsmqj"
Oct 30 21:58:59.301334 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:59.301312    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cc269490-09f2-4b2b-ba6d-df674223d731-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3bdc7244-nsmqj\" (UID: \"cc269490-09f2-4b2b-ba6d-df674223d731\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3bdc7244-nsmqj"
Oct 30 21:58:59.357557 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:59.357522    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2sfl\" (UniqueName: \"kubernetes.io/projected/cc269490-09f2-4b2b-ba6d-df674223d731-kube-api-access-g2sfl\") pod \"maas-api-revoke-sub-3bdc7244-nsmqj\" (UID: \"cc269490-09f2-4b2b-ba6d-df674223d731\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3bdc7244-nsmqj"
Oct 30 21:58:59.440674 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:59.440582    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3bdc7244-nsmqj"
Oct 30 21:58:59.818083 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:59.818045    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3bdc7244-nsmqj"]
Oct 30 21:58:59.819851 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:58:59.819812    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc269490_09f2_4b2b_ba6d_df674223d731.slice/crio-33c7a52677a7fc0fdaa80f2c55d08f73cc9c43fbf50920b136c8ffe6c8ca5aaf WatchSource:0}: Error finding container 33c7a52677a7fc0fdaa80f2c55d08f73cc9c43fbf50920b136c8ffe6c8ca5aaf: Status 404 returned error can't find the container with id 33c7a52677a7fc0fdaa80f2c55d08f73cc9c43fbf50920b136c8ffe6c8ca5aaf
Oct 30 21:58:59.999700 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:58:59.999658    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3bdc7244-nsmqj" event={"ID":"cc269490-09f2-4b2b-ba6d-df674223d731","Type":"ContainerStarted","Data":"33c7a52677a7fc0fdaa80f2c55d08f73cc9c43fbf50920b136c8ffe6c8ca5aaf"}
Oct 30 21:59:00.831950 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:00.831891    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:01.010192 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:01.010153    4871 generic.go:356] "Generic (PLEG): container finished" podID="cc269490-09f2-4b2b-ba6d-df674223d731" containerID="74bafa3b50d112fcee4591e0ae3ec2df8e1b315731ffd4c74b4954ac58623da7" exitCode=0
Oct 30 21:59:01.010397 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:01.010256    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3bdc7244-nsmqj" event={"ID":"cc269490-09f2-4b2b-ba6d-df674223d731","Type":"ContainerDied","Data":"74bafa3b50d112fcee4591e0ae3ec2df8e1b315731ffd4c74b4954ac58623da7"}
Oct 30 21:59:02.191277 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:02.191250    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3bdc7244-nsmqj"
Oct 30 21:59:02.337014 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:02.336914    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cc269490-09f2-4b2b-ba6d-df674223d731-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cc269490-09f2-4b2b-ba6d-df674223d731-maas-api-openshift-ca\") pod \"cc269490-09f2-4b2b-ba6d-df674223d731\" (UID: \"cc269490-09f2-4b2b-ba6d-df674223d731\") "
Oct 30 21:59:02.337014 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:02.336968    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cc269490-09f2-4b2b-ba6d-df674223d731-kube-api-access-g2sfl\" (UniqueName: \"kubernetes.io/projected/cc269490-09f2-4b2b-ba6d-df674223d731-kube-api-access-g2sfl\") pod \"cc269490-09f2-4b2b-ba6d-df674223d731\" (UID: \"cc269490-09f2-4b2b-ba6d-df674223d731\") "
Oct 30 21:59:02.337272 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:02.337107    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cc269490-09f2-4b2b-ba6d-df674223d731-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cc269490-09f2-4b2b-ba6d-df674223d731-maas-api-xks-ca\") pod \"cc269490-09f2-4b2b-ba6d-df674223d731\" (UID: \"cc269490-09f2-4b2b-ba6d-df674223d731\") "
Oct 30 21:59:02.337406 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:02.337375    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc269490-09f2-4b2b-ba6d-df674223d731-maas-api-openshift-ca" pod "cc269490-09f2-4b2b-ba6d-df674223d731" (UID: "cc269490-09f2-4b2b-ba6d-df674223d731"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:59:02.337680 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:02.337660    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cc269490-09f2-4b2b-ba6d-df674223d731-maas-api-openshift-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:02.339323 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:02.339293    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc269490-09f2-4b2b-ba6d-df674223d731-kube-api-access-g2sfl" pod "cc269490-09f2-4b2b-ba6d-df674223d731" (UID: "cc269490-09f2-4b2b-ba6d-df674223d731"). InnerVolumeSpecName "kube-api-access-g2sfl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:59:02.339569 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:02.339548    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc269490-09f2-4b2b-ba6d-df674223d731-maas-api-xks-ca" pod "cc269490-09f2-4b2b-ba6d-df674223d731" (UID: "cc269490-09f2-4b2b-ba6d-df674223d731"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:59:02.438794 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:02.438754    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cc269490-09f2-4b2b-ba6d-df674223d731-maas-api-xks-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:02.438794 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:02.438788    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2sfl\" (UniqueName: \"kubernetes.io/projected/cc269490-09f2-4b2b-ba6d-df674223d731-kube-api-access-g2sfl\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:03.029567 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:03.029527    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3bdc7244-nsmqj"
Oct 30 21:59:03.029754 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:03.029525    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3bdc7244-nsmqj" event={"ID":"cc269490-09f2-4b2b-ba6d-df674223d731","Type":"ContainerDied","Data":"33c7a52677a7fc0fdaa80f2c55d08f73cc9c43fbf50920b136c8ffe6c8ca5aaf"}
Oct 30 21:59:03.029754 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:03.029678    4871 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="33c7a52677a7fc0fdaa80f2c55d08f73cc9c43fbf50920b136c8ffe6c8ca5aaf"
Oct 30 21:59:05.800546 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:59:05.800459    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:59:06.168886 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:06.123892    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:10.224288 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:10.223042    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:10.412378 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:10.412341    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:10.646276 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:10.639835    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:11.240889 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:11.226525    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:11.309298 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:11.309258    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-866b9fd699-kvmpl"]
Oct 30 21:59:11.309907 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:11.309889    4871 cpu_manager.go:414] "containerMap: removing container" podUID="cc269490-09f2-4b2b-ba6d-df674223d731" containerName="revoke-keys"
Oct 30 21:59:11.309907 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:11.309906    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cc269490-09f2-4b2b-ba6d-df674223d731" containerName="revoke-keys"
Oct 30 21:59:11.310059 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:11.310009    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="cc269490-09f2-4b2b-ba6d-df674223d731" containerName="revoke-keys"
Oct 30 21:59:11.314875 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:11.314854    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-866b9fd699-kvmpl"
Oct 30 21:59:11.371589 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:11.371555    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-866b9fd699-kvmpl"]
Oct 30 21:59:11.427463 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:11.427417    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/7c3f8565-5b8f-4709-bacb-773e5abc4659-provider-credentials\") pod \"payload-processing-external-model-866b9fd699-kvmpl\" (UID: \"7c3f8565-5b8f-4709-bacb-773e5abc4659\") " pod="models-as-a-service/payload-processing-external-model-866b9fd699-kvmpl"
Oct 30 21:59:11.427645 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:11.427485    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c54r9\" (UniqueName: \"kubernetes.io/projected/7c3f8565-5b8f-4709-bacb-773e5abc4659-kube-api-access-c54r9\") pod \"payload-processing-external-model-866b9fd699-kvmpl\" (UID: \"7c3f8565-5b8f-4709-bacb-773e5abc4659\") " pod="models-as-a-service/payload-processing-external-model-866b9fd699-kvmpl"
Oct 30 21:59:11.427695 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:11.427664    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/7c3f8565-5b8f-4709-bacb-773e5abc4659-routing-overlay-volume\") pod \"payload-processing-external-model-866b9fd699-kvmpl\" (UID: \"7c3f8565-5b8f-4709-bacb-773e5abc4659\") " pod="models-as-a-service/payload-processing-external-model-866b9fd699-kvmpl"
Oct 30 21:59:11.427739 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:11.427706    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c3f8565-5b8f-4709-bacb-773e5abc4659-plugins-config-volume\") pod \"payload-processing-external-model-866b9fd699-kvmpl\" (UID: \"7c3f8565-5b8f-4709-bacb-773e5abc4659\") " pod="models-as-a-service/payload-processing-external-model-866b9fd699-kvmpl"
Oct 30 21:59:11.528827 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:11.528721    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/7c3f8565-5b8f-4709-bacb-773e5abc4659-provider-credentials\") pod \"payload-processing-external-model-866b9fd699-kvmpl\" (UID: \"7c3f8565-5b8f-4709-bacb-773e5abc4659\") " pod="models-as-a-service/payload-processing-external-model-866b9fd699-kvmpl"
Oct 30 21:59:11.529010 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:11.528862    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c54r9\" (UniqueName: \"kubernetes.io/projected/7c3f8565-5b8f-4709-bacb-773e5abc4659-kube-api-access-c54r9\") pod \"payload-processing-external-model-866b9fd699-kvmpl\" (UID: \"7c3f8565-5b8f-4709-bacb-773e5abc4659\") " pod="models-as-a-service/payload-processing-external-model-866b9fd699-kvmpl"
Oct 30 21:59:11.529010 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:11.528984    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/7c3f8565-5b8f-4709-bacb-773e5abc4659-routing-overlay-volume\") pod \"payload-processing-external-model-866b9fd699-kvmpl\" (UID: \"7c3f8565-5b8f-4709-bacb-773e5abc4659\") " pod="models-as-a-service/payload-processing-external-model-866b9fd699-kvmpl"
Oct 30 21:59:11.529127 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:11.529014    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c3f8565-5b8f-4709-bacb-773e5abc4659-plugins-config-volume\") pod \"payload-processing-external-model-866b9fd699-kvmpl\" (UID: \"7c3f8565-5b8f-4709-bacb-773e5abc4659\") " pod="models-as-a-service/payload-processing-external-model-866b9fd699-kvmpl"
Oct 30 21:59:11.529835 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:11.529814    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c3f8565-5b8f-4709-bacb-773e5abc4659-plugins-config-volume\") pod \"payload-processing-external-model-866b9fd699-kvmpl\" (UID: \"7c3f8565-5b8f-4709-bacb-773e5abc4659\") " pod="models-as-a-service/payload-processing-external-model-866b9fd699-kvmpl"
Oct 30 21:59:11.529916 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:11.529817    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/7c3f8565-5b8f-4709-bacb-773e5abc4659-routing-overlay-volume\") pod \"payload-processing-external-model-866b9fd699-kvmpl\" (UID: \"7c3f8565-5b8f-4709-bacb-773e5abc4659\") " pod="models-as-a-service/payload-processing-external-model-866b9fd699-kvmpl"
Oct 30 21:59:11.531180 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:11.531155    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/7c3f8565-5b8f-4709-bacb-773e5abc4659-provider-credentials\") pod \"payload-processing-external-model-866b9fd699-kvmpl\" (UID: \"7c3f8565-5b8f-4709-bacb-773e5abc4659\") " pod="models-as-a-service/payload-processing-external-model-866b9fd699-kvmpl"
Oct 30 21:59:11.607530 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:11.607489    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c54r9\" (UniqueName: \"kubernetes.io/projected/7c3f8565-5b8f-4709-bacb-773e5abc4659-kube-api-access-c54r9\") pod \"payload-processing-external-model-866b9fd699-kvmpl\" (UID: \"7c3f8565-5b8f-4709-bacb-773e5abc4659\") " pod="models-as-a-service/payload-processing-external-model-866b9fd699-kvmpl"
Oct 30 21:59:11.630360 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:11.630322    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-866b9fd699-kvmpl"
Oct 30 21:59:12.022743 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:12.022614    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-866b9fd699-kvmpl"]
Oct 30 21:59:12.025837 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:59:12.025801    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c3f8565_5b8f_4709_bacb_773e5abc4659.slice/crio-fcb17ea4d6d400b674fd54a5935a017a5423fc644b294bc146758097acfc860d WatchSource:0}: Error finding container fcb17ea4d6d400b674fd54a5935a017a5423fc644b294bc146758097acfc860d: Status 404 returned error can't find the container with id fcb17ea4d6d400b674fd54a5935a017a5423fc644b294bc146758097acfc860d
Oct 30 21:59:12.277838 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:12.277745    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-866b9fd699-kvmpl" event={"ID":"7c3f8565-5b8f-4709-bacb-773e5abc4659","Type":"ContainerStarted","Data":"dd1c30fbe58d957e68c1f3ea86a425c2f56141eee3607dda8d4d1a0f3380ff37"}
Oct 30 21:59:12.277838 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:12.277807    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-866b9fd699-kvmpl"
Oct 30 21:59:12.277838 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:12.277824    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-866b9fd699-kvmpl" event={"ID":"7c3f8565-5b8f-4709-bacb-773e5abc4659","Type":"ContainerStarted","Data":"fcb17ea4d6d400b674fd54a5935a017a5423fc644b294bc146758097acfc860d"}
Oct 30 21:59:12.401559 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:12.401484    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-866b9fd699-kvmpl" podStartSLOduration=1.4014669419999999 podStartE2EDuration="1.401466942s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:59:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:59:12.398353291 +0000 UTC m=+1294.575841908" watchObservedRunningTime="2026-10-30 21:59:12.401466942 +0000 UTC m=+1294.578955557"
Oct 30 21:59:15.944723 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:15.944685    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:20.668012 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:20.667783    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:22.997048 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:59:22.997005    4871 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-00e4c0ba7157: secret "agc-cred-00e4c0ba7157" not found
Oct 30 21:59:22.997048 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:59:22.997045    4871 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-3d0f8113fe57: secret "agc-cred-3d0f8113fe57" not found
Oct 30 21:59:22.997663 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:59:22.997067    4871 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-78677bf7-fm4j7: [secret "agc-cred-00e4c0ba7157" not found, secret "agc-cred-3d0f8113fe57" not found]
Oct 30 21:59:22.997663 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:59:22.997144    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/33d005ac-b847-4fd5-af8f-5b0162b56610-provider-credentials podName:33d005ac-b847-4fd5-af8f-5b0162b56610 nodeName:}" failed. No retries permitted until 2026-10-30 21:59:23.497124023 +0000 UTC m=+1305.674612619 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/33d005ac-b847-4fd5-af8f-5b0162b56610-provider-credentials") pod "payload-processing-external-model-78677bf7-fm4j7" (UID: "33d005ac-b847-4fd5-af8f-5b0162b56610") : [secret "agc-cred-00e4c0ba7157" not found, secret "agc-cred-3d0f8113fe57" not found]
Oct 30 21:59:23.292044 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:23.291951    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-866b9fd699-kvmpl" podUID="7c3f8565-5b8f-4709-bacb-773e5abc4659" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:59:23.508632 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:59:23.508594    4871 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-00e4c0ba7157: secret "agc-cred-00e4c0ba7157" not found
Oct 30 21:59:23.508632 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:59:23.508625    4871 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-3d0f8113fe57: secret "agc-cred-3d0f8113fe57" not found
Oct 30 21:59:23.508632 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:59:23.508643    4871 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-78677bf7-fm4j7: [secret "agc-cred-00e4c0ba7157" not found, secret "agc-cred-3d0f8113fe57" not found]
Oct 30 21:59:23.508990 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:59:23.508719    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/33d005ac-b847-4fd5-af8f-5b0162b56610-provider-credentials podName:33d005ac-b847-4fd5-af8f-5b0162b56610 nodeName:}" failed. No retries permitted until 2026-10-30 21:59:24.508701024 +0000 UTC m=+1306.686189632 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/33d005ac-b847-4fd5-af8f-5b0162b56610-provider-credentials") pod "payload-processing-external-model-78677bf7-fm4j7" (UID: "33d005ac-b847-4fd5-af8f-5b0162b56610") : [secret "agc-cred-00e4c0ba7157" not found, secret "agc-cred-3d0f8113fe57" not found]
Oct 30 21:59:23.957260 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:23.938703    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:24.099426 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:24.099385    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:24.525616 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:59:24.525573    4871 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-00e4c0ba7157: secret "agc-cred-00e4c0ba7157" not found
Oct 30 21:59:24.525616 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:59:24.525610    4871 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-3d0f8113fe57: secret "agc-cred-3d0f8113fe57" not found
Oct 30 21:59:24.525616 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:59:24.525619    4871 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-7653478ffe43: secret "agc-cred-7653478ffe43" not found
Oct 30 21:59:24.525877 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:59:24.525632    4871 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-78677bf7-fm4j7: [secret "agc-cred-00e4c0ba7157" not found, secret "agc-cred-3d0f8113fe57" not found, secret "agc-cred-7653478ffe43" not found]
Oct 30 21:59:24.525877 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:59:24.525708    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/33d005ac-b847-4fd5-af8f-5b0162b56610-provider-credentials podName:33d005ac-b847-4fd5-af8f-5b0162b56610 nodeName:}" failed. No retries permitted until 2026-10-30 21:59:26.525687301 +0000 UTC m=+1308.703175916 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/33d005ac-b847-4fd5-af8f-5b0162b56610-provider-credentials") pod "payload-processing-external-model-78677bf7-fm4j7" (UID: "33d005ac-b847-4fd5-af8f-5b0162b56610") : [secret "agc-cred-00e4c0ba7157" not found, secret "agc-cred-3d0f8113fe57" not found, secret "agc-cred-7653478ffe43" not found]
Oct 30 21:59:25.294602 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:25.294569    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:25.459035 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:25.458995    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"]
Oct 30 21:59:25.464655 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:25.464628    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"
Oct 30 21:59:25.549770 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:25.549679    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"]
Oct 30 21:59:25.558499 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:25.558463    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85541021-4964-4ce1-b471-9df326efc190-workload-certs\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2\" (UID: \"85541021-4964-4ce1-b471-9df326efc190\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"
Oct 30 21:59:25.559068 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:25.558586    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85541021-4964-4ce1-b471-9df326efc190-istio-envoy\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2\" (UID: \"85541021-4964-4ce1-b471-9df326efc190\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"
Oct 30 21:59:25.559068 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:25.558617    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85541021-4964-4ce1-b471-9df326efc190-workload-socket\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2\" (UID: \"85541021-4964-4ce1-b471-9df326efc190\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"
Oct 30 21:59:25.559068 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:25.558636    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85541021-4964-4ce1-b471-9df326efc190-istio-token\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2\" (UID: \"85541021-4964-4ce1-b471-9df326efc190\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"
Oct 30 21:59:25.559068 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:25.558653    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85541021-4964-4ce1-b471-9df326efc190-istio-podinfo\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2\" (UID: \"85541021-4964-4ce1-b471-9df326efc190\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"
Oct 30 21:59:25.559068 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:25.558682    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsxkc\" (UniqueName: \"kubernetes.io/projected/85541021-4964-4ce1-b471-9df326efc190-kube-api-access-tsxkc\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2\" (UID: \"85541021-4964-4ce1-b471-9df326efc190\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"
Oct 30 21:59:25.559068 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:25.558718    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85541021-4964-4ce1-b471-9df326efc190-istio-data\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2\" (UID: \"85541021-4964-4ce1-b471-9df326efc190\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"
Oct 30 21:59:25.559068 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:25.558739    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85541021-4964-4ce1-b471-9df326efc190-credential-socket\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2\" (UID: \"85541021-4964-4ce1-b471-9df326efc190\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"
Oct 30 21:59:25.559068 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:25.558770    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85541021-4964-4ce1-b471-9df326efc190-istiod-ca-cert\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2\" (UID: \"85541021-4964-4ce1-b471-9df326efc190\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"
Oct 30 21:59:25.659629 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:25.659582    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85541021-4964-4ce1-b471-9df326efc190-istio-data\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2\" (UID: \"85541021-4964-4ce1-b471-9df326efc190\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"
Oct 30 21:59:25.659629 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:25.659636    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85541021-4964-4ce1-b471-9df326efc190-credential-socket\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2\" (UID: \"85541021-4964-4ce1-b471-9df326efc190\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"
Oct 30 21:59:25.659901 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:25.659665    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85541021-4964-4ce1-b471-9df326efc190-istiod-ca-cert\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2\" (UID: \"85541021-4964-4ce1-b471-9df326efc190\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"
Oct 30 21:59:25.659901 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:25.659719    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85541021-4964-4ce1-b471-9df326efc190-workload-certs\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2\" (UID: \"85541021-4964-4ce1-b471-9df326efc190\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"
Oct 30 21:59:25.659901 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:25.659878    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85541021-4964-4ce1-b471-9df326efc190-istio-envoy\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2\" (UID: \"85541021-4964-4ce1-b471-9df326efc190\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"
Oct 30 21:59:25.660062 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:25.659924    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85541021-4964-4ce1-b471-9df326efc190-workload-socket\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2\" (UID: \"85541021-4964-4ce1-b471-9df326efc190\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"
Oct 30 21:59:25.660062 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:25.659952    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85541021-4964-4ce1-b471-9df326efc190-istio-token\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2\" (UID: \"85541021-4964-4ce1-b471-9df326efc190\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"
Oct 30 21:59:25.660062 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:25.659979    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85541021-4964-4ce1-b471-9df326efc190-istio-podinfo\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2\" (UID: \"85541021-4964-4ce1-b471-9df326efc190\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"
Oct 30 21:59:25.660062 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:25.660032    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsxkc\" (UniqueName: \"kubernetes.io/projected/85541021-4964-4ce1-b471-9df326efc190-kube-api-access-tsxkc\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2\" (UID: \"85541021-4964-4ce1-b471-9df326efc190\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"
Oct 30 21:59:25.660062 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:25.660038    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85541021-4964-4ce1-b471-9df326efc190-istio-data\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2\" (UID: \"85541021-4964-4ce1-b471-9df326efc190\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"
Oct 30 21:59:25.660322 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:25.660099    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85541021-4964-4ce1-b471-9df326efc190-credential-socket\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2\" (UID: \"85541021-4964-4ce1-b471-9df326efc190\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"
Oct 30 21:59:25.660322 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:25.660146    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85541021-4964-4ce1-b471-9df326efc190-workload-certs\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2\" (UID: \"85541021-4964-4ce1-b471-9df326efc190\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"
Oct 30 21:59:25.660322 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:25.660309    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85541021-4964-4ce1-b471-9df326efc190-workload-socket\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2\" (UID: \"85541021-4964-4ce1-b471-9df326efc190\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"
Oct 30 21:59:25.660716 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:25.660696    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85541021-4964-4ce1-b471-9df326efc190-istiod-ca-cert\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2\" (UID: \"85541021-4964-4ce1-b471-9df326efc190\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"
Oct 30 21:59:25.662276 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:25.662250    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85541021-4964-4ce1-b471-9df326efc190-istio-envoy\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2\" (UID: \"85541021-4964-4ce1-b471-9df326efc190\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"
Oct 30 21:59:25.662559 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:25.662539    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85541021-4964-4ce1-b471-9df326efc190-istio-podinfo\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2\" (UID: \"85541021-4964-4ce1-b471-9df326efc190\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"
Oct 30 21:59:25.718718 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:25.718682    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85541021-4964-4ce1-b471-9df326efc190-istio-token\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2\" (UID: \"85541021-4964-4ce1-b471-9df326efc190\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"
Oct 30 21:59:25.718904 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:25.718827    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsxkc\" (UniqueName: \"kubernetes.io/projected/85541021-4964-4ce1-b471-9df326efc190-kube-api-access-tsxkc\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2\" (UID: \"85541021-4964-4ce1-b471-9df326efc190\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"
Oct 30 21:59:25.781072 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:25.781029    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"
Oct 30 21:59:26.219477 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:26.219441    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"]
Oct 30 21:59:26.221016 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:59:26.220977    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85541021_4964_4ce1_b471_9df326efc190.slice/crio-ab97931147a1aa7bb2f9fc2d18f0812ca52b905b4ef9bbf9ab3219dbf08407c7 WatchSource:0}: Error finding container ab97931147a1aa7bb2f9fc2d18f0812ca52b905b4ef9bbf9ab3219dbf08407c7: Status 404 returned error can't find the container with id ab97931147a1aa7bb2f9fc2d18f0812ca52b905b4ef9bbf9ab3219dbf08407c7
Oct 30 21:59:26.223529 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:26.223492    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:59:26.223640 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:26.223559    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:59:26.223640 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:26.223586    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:59:26.237405 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:26.237369    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-78677bf7-fm4j7"]
Oct 30 21:59:26.265021 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:26.264989    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-866b9fd699-kvmpl"]
Oct 30 21:59:26.266450 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:26.266422    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-866b9fd699-kvmpl" podUID="7c3f8565-5b8f-4709-bacb-773e5abc4659" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:59:26.405692 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:26.405649    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2" event={"ID":"85541021-4964-4ce1-b471-9df326efc190","Type":"ContainerStarted","Data":"eb571be43aba8fc65f6de5098c2a28fa0a8caeaffaf7ca3252034053c1e7e1d1"}
Oct 30 21:59:26.406190 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:26.405710    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2" event={"ID":"85541021-4964-4ce1-b471-9df326efc190","Type":"ContainerStarted","Data":"ab97931147a1aa7bb2f9fc2d18f0812ca52b905b4ef9bbf9ab3219dbf08407c7"}
Oct 30 21:59:26.517327 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:26.515512    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:26.583341 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:59:26.583306    4871 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-00e4c0ba7157: secret "agc-cred-00e4c0ba7157" not found
Oct 30 21:59:26.583341 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:59:26.583337    4871 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-3d0f8113fe57: secret "agc-cred-3d0f8113fe57" not found
Oct 30 21:59:26.583341 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:59:26.583344    4871 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-7653478ffe43: secret "agc-cred-7653478ffe43" not found
Oct 30 21:59:26.583341 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:59:26.583353    4871 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-78677bf7-fm4j7: [secret "agc-cred-00e4c0ba7157" not found, secret "agc-cred-3d0f8113fe57" not found, secret "agc-cred-7653478ffe43" not found]
Oct 30 21:59:26.583698 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:59:26.583408    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/33d005ac-b847-4fd5-af8f-5b0162b56610-provider-credentials podName:33d005ac-b847-4fd5-af8f-5b0162b56610 nodeName:}" failed. No retries permitted until 2026-10-30 21:59:30.583393798 +0000 UTC m=+1312.760882405 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/33d005ac-b847-4fd5-af8f-5b0162b56610-provider-credentials") pod "payload-processing-external-model-78677bf7-fm4j7" (UID: "33d005ac-b847-4fd5-af8f-5b0162b56610") : [secret "agc-cred-00e4c0ba7157" not found, secret "agc-cred-3d0f8113fe57" not found, secret "agc-cred-7653478ffe43" not found]
Oct 30 21:59:26.627308 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:26.627223    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2" podStartSLOduration=1.627203786 podStartE2EDuration="1.627203786s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:59:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:59:26.625558595 +0000 UTC m=+1308.803047214" watchObservedRunningTime="2026-10-30 21:59:26.627203786 +0000 UTC m=+1308.804692403"
Oct 30 21:59:26.781770 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:26.781682    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"
Oct 30 21:59:27.243259 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:27.241568    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"]
Oct 30 21:59:27.268264 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:27.263665    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"
Oct 30 21:59:27.331257 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:27.324176    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"]
Oct 30 21:59:27.417149 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:27.408896    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-istio-data\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt\" (UID: \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"
Oct 30 21:59:27.417149 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:27.409032    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-istio-token\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt\" (UID: \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"
Oct 30 21:59:27.417149 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:27.409087    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-credential-socket\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt\" (UID: \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"
Oct 30 21:59:27.417149 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:27.409132    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-workload-socket\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt\" (UID: \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"
Oct 30 21:59:27.417149 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:27.409215    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2x2z6\" (UniqueName: \"kubernetes.io/projected/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-kube-api-access-2x2z6\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt\" (UID: \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"
Oct 30 21:59:27.417149 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:27.409312    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-istio-podinfo\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt\" (UID: \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"
Oct 30 21:59:27.417149 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:27.409394    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-workload-certs\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt\" (UID: \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"
Oct 30 21:59:27.417149 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:27.409439    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-istio-envoy\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt\" (UID: \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"
Oct 30 21:59:27.417149 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:27.409585    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-istiod-ca-cert\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt\" (UID: \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"
Oct 30 21:59:27.524326 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:27.514340    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-istiod-ca-cert\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt\" (UID: \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"
Oct 30 21:59:27.524326 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:27.514465    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-istio-data\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt\" (UID: \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"
Oct 30 21:59:27.524326 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:27.514520    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-istio-token\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt\" (UID: \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"
Oct 30 21:59:27.524326 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:27.514547    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-credential-socket\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt\" (UID: \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"
Oct 30 21:59:27.524326 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:27.514587    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-workload-socket\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt\" (UID: \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"
Oct 30 21:59:27.524326 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:27.514672    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2x2z6\" (UniqueName: \"kubernetes.io/projected/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-kube-api-access-2x2z6\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt\" (UID: \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"
Oct 30 21:59:27.524326 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:27.514739    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-istio-podinfo\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt\" (UID: \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"
Oct 30 21:59:27.524326 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:27.514808    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-workload-certs\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt\" (UID: \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"
Oct 30 21:59:27.524326 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:27.514863    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-istio-envoy\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt\" (UID: \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"
Oct 30 21:59:27.528259 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:27.526160    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-credential-socket\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt\" (UID: \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"
Oct 30 21:59:27.532520 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:27.532475    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-istiod-ca-cert\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt\" (UID: \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"
Oct 30 21:59:27.540587 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:27.540538    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-workload-socket\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt\" (UID: \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"
Oct 30 21:59:27.540753 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:27.540615    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-istio-envoy\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt\" (UID: \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"
Oct 30 21:59:27.540753 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:27.540621    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-workload-certs\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt\" (UID: \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"
Oct 30 21:59:27.540831 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:27.540794    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-istio-data\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt\" (UID: \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"
Oct 30 21:59:27.543380 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:27.543355    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-istio-podinfo\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt\" (UID: \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"
Oct 30 21:59:27.583926 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:27.583880    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-istio-token\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt\" (UID: \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"
Oct 30 21:59:27.584322 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:27.584302    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2x2z6\" (UniqueName: \"kubernetes.io/projected/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-kube-api-access-2x2z6\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt\" (UID: \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"
Oct 30 21:59:27.624262 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:27.610512    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"
Oct 30 21:59:27.783640 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:27.783546    4871 patch_prober.go:29] interesting pod/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 21:59:27.783640 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:27.783618    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2" podUID="85541021-4964-4ce1-b471-9df326efc190" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 21:59:28.066417 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:28.066371    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"]
Oct 30 21:59:28.070228 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:28.070182    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:59:28.070361 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:28.070316    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:59:28.070423 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:28.070365    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:59:28.446894 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:28.446852    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt" event={"ID":"52b9b4dc-149f-4650-81f7-7f0234d9c6d8","Type":"ContainerStarted","Data":"cd2786aec1e8db07dcd8ecd2304de1e7078e3f076a7fd2cc0b95628e9f179756"}
Oct 30 21:59:28.447428 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:28.446937    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt" event={"ID":"52b9b4dc-149f-4650-81f7-7f0234d9c6d8","Type":"ContainerStarted","Data":"a14292423c58d043d91759649fc807d8fd8cef4e136d3b47bf0869216aa98f4b"}
Oct 30 21:59:28.562996 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:28.562937    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt" podStartSLOduration=1.5629204049999998 podStartE2EDuration="1.562920405s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:59:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:59:28.560444965 +0000 UTC m=+1310.737933582" watchObservedRunningTime="2026-10-30 21:59:28.562920405 +0000 UTC m=+1310.740409021"
Oct 30 21:59:28.631262 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:28.630373    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"
Oct 30 21:59:28.782296 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:28.782171    4871 patch_prober.go:29] interesting pod/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 21:59:28.782296 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:28.782270    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2" podUID="85541021-4964-4ce1-b471-9df326efc190" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 21:59:29.309260 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.301947    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"]
Oct 30 21:59:29.365262 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.364597    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"
Oct 30 21:59:29.373262 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.366377    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"]
Oct 30 21:59:29.387258 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.385680    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47d9d464-c7c2-4980-ba4d-e3320373ed88-istio-envoy\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh\" (UID: \"47d9d464-c7c2-4980-ba4d-e3320373ed88\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"
Oct 30 21:59:29.387258 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.385759    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47d9d464-c7c2-4980-ba4d-e3320373ed88-istiod-ca-cert\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh\" (UID: \"47d9d464-c7c2-4980-ba4d-e3320373ed88\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"
Oct 30 21:59:29.387258 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.385805    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47d9d464-c7c2-4980-ba4d-e3320373ed88-credential-socket\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh\" (UID: \"47d9d464-c7c2-4980-ba4d-e3320373ed88\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"
Oct 30 21:59:29.387258 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.385840    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47d9d464-c7c2-4980-ba4d-e3320373ed88-istio-podinfo\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh\" (UID: \"47d9d464-c7c2-4980-ba4d-e3320373ed88\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"
Oct 30 21:59:29.387258 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.385868    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47d9d464-c7c2-4980-ba4d-e3320373ed88-istio-data\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh\" (UID: \"47d9d464-c7c2-4980-ba4d-e3320373ed88\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"
Oct 30 21:59:29.387258 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.385946    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47d9d464-c7c2-4980-ba4d-e3320373ed88-workload-socket\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh\" (UID: \"47d9d464-c7c2-4980-ba4d-e3320373ed88\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"
Oct 30 21:59:29.387258 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.385995    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47d9d464-c7c2-4980-ba4d-e3320373ed88-istio-token\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh\" (UID: \"47d9d464-c7c2-4980-ba4d-e3320373ed88\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"
Oct 30 21:59:29.387258 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.386061    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47d9d464-c7c2-4980-ba4d-e3320373ed88-workload-certs\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh\" (UID: \"47d9d464-c7c2-4980-ba4d-e3320373ed88\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"
Oct 30 21:59:29.387258 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.386140    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bspwc\" (UniqueName: \"kubernetes.io/projected/47d9d464-c7c2-4980-ba4d-e3320373ed88-kube-api-access-bspwc\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh\" (UID: \"47d9d464-c7c2-4980-ba4d-e3320373ed88\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"
Oct 30 21:59:29.436275 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.429811    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"
Oct 30 21:59:29.495259 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.472450    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"]
Oct 30 21:59:29.495259 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.487894    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f506e56e-854d-4a96-a255-f04c7c31a57d-workload-certs\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc\" (UID: \"f506e56e-854d-4a96-a255-f04c7c31a57d\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"
Oct 30 21:59:29.495259 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.487941    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f506e56e-854d-4a96-a255-f04c7c31a57d-istio-envoy\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc\" (UID: \"f506e56e-854d-4a96-a255-f04c7c31a57d\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"
Oct 30 21:59:29.495259 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.487973    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47d9d464-c7c2-4980-ba4d-e3320373ed88-credential-socket\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh\" (UID: \"47d9d464-c7c2-4980-ba4d-e3320373ed88\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"
Oct 30 21:59:29.495259 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.488001    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47d9d464-c7c2-4980-ba4d-e3320373ed88-istio-podinfo\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh\" (UID: \"47d9d464-c7c2-4980-ba4d-e3320373ed88\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"
Oct 30 21:59:29.495259 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.488033    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47d9d464-c7c2-4980-ba4d-e3320373ed88-istio-data\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh\" (UID: \"47d9d464-c7c2-4980-ba4d-e3320373ed88\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"
Oct 30 21:59:29.495259 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.488076    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f506e56e-854d-4a96-a255-f04c7c31a57d-istio-data\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc\" (UID: \"f506e56e-854d-4a96-a255-f04c7c31a57d\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"
Oct 30 21:59:29.495259 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.488134    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47d9d464-c7c2-4980-ba4d-e3320373ed88-workload-socket\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh\" (UID: \"47d9d464-c7c2-4980-ba4d-e3320373ed88\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"
Oct 30 21:59:29.495259 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.488162    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f506e56e-854d-4a96-a255-f04c7c31a57d-istio-podinfo\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc\" (UID: \"f506e56e-854d-4a96-a255-f04c7c31a57d\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"
Oct 30 21:59:29.495259 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.488187    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f506e56e-854d-4a96-a255-f04c7c31a57d-istio-token\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc\" (UID: \"f506e56e-854d-4a96-a255-f04c7c31a57d\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"
Oct 30 21:59:29.518393 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.513387    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47d9d464-c7c2-4980-ba4d-e3320373ed88-istio-token\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh\" (UID: \"47d9d464-c7c2-4980-ba4d-e3320373ed88\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"
Oct 30 21:59:29.518393 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.513480    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f506e56e-854d-4a96-a255-f04c7c31a57d-workload-socket\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc\" (UID: \"f506e56e-854d-4a96-a255-f04c7c31a57d\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"
Oct 30 21:59:29.518393 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.513617    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f506e56e-854d-4a96-a255-f04c7c31a57d-credential-socket\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc\" (UID: \"f506e56e-854d-4a96-a255-f04c7c31a57d\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"
Oct 30 21:59:29.518393 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.513654    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47d9d464-c7c2-4980-ba4d-e3320373ed88-workload-certs\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh\" (UID: \"47d9d464-c7c2-4980-ba4d-e3320373ed88\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"
Oct 30 21:59:29.518393 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.513770    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5whcd\" (UniqueName: \"kubernetes.io/projected/f506e56e-854d-4a96-a255-f04c7c31a57d-kube-api-access-5whcd\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc\" (UID: \"f506e56e-854d-4a96-a255-f04c7c31a57d\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"
Oct 30 21:59:29.518393 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.513875    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f506e56e-854d-4a96-a255-f04c7c31a57d-istiod-ca-cert\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc\" (UID: \"f506e56e-854d-4a96-a255-f04c7c31a57d\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"
Oct 30 21:59:29.518393 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.513924    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bspwc\" (UniqueName: \"kubernetes.io/projected/47d9d464-c7c2-4980-ba4d-e3320373ed88-kube-api-access-bspwc\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh\" (UID: \"47d9d464-c7c2-4980-ba4d-e3320373ed88\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"
Oct 30 21:59:29.518393 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.514016    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47d9d464-c7c2-4980-ba4d-e3320373ed88-istio-envoy\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh\" (UID: \"47d9d464-c7c2-4980-ba4d-e3320373ed88\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"
Oct 30 21:59:29.518393 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.514087    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47d9d464-c7c2-4980-ba4d-e3320373ed88-istiod-ca-cert\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh\" (UID: \"47d9d464-c7c2-4980-ba4d-e3320373ed88\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"
Oct 30 21:59:29.518393 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.517050    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47d9d464-c7c2-4980-ba4d-e3320373ed88-credential-socket\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh\" (UID: \"47d9d464-c7c2-4980-ba4d-e3320373ed88\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"
Oct 30 21:59:29.553779 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.538211    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47d9d464-c7c2-4980-ba4d-e3320373ed88-workload-certs\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh\" (UID: \"47d9d464-c7c2-4980-ba4d-e3320373ed88\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"
Oct 30 21:59:29.553779 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.538787    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47d9d464-c7c2-4980-ba4d-e3320373ed88-workload-socket\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh\" (UID: \"47d9d464-c7c2-4980-ba4d-e3320373ed88\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"
Oct 30 21:59:29.553779 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.539018    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47d9d464-c7c2-4980-ba4d-e3320373ed88-istio-data\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh\" (UID: \"47d9d464-c7c2-4980-ba4d-e3320373ed88\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"
Oct 30 21:59:29.582912 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.576782    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"]
Oct 30 21:59:29.624859 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.587604    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47d9d464-c7c2-4980-ba4d-e3320373ed88-istiod-ca-cert\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh\" (UID: \"47d9d464-c7c2-4980-ba4d-e3320373ed88\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"
Oct 30 21:59:29.650294 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.650224    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 30 21:59:29.650565 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.650532    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt" podUID="52b9b4dc-149f-4650-81f7-7f0234d9c6d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 30 21:59:29.765455 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.737392    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:29.765455 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.738407    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f506e56e-854d-4a96-a255-f04c7c31a57d-credential-socket\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc\" (UID: \"f506e56e-854d-4a96-a255-f04c7c31a57d\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"
Oct 30 21:59:29.765455 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.738489    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5whcd\" (UniqueName: \"kubernetes.io/projected/f506e56e-854d-4a96-a255-f04c7c31a57d-kube-api-access-5whcd\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc\" (UID: \"f506e56e-854d-4a96-a255-f04c7c31a57d\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"
Oct 30 21:59:29.765455 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.738559    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f506e56e-854d-4a96-a255-f04c7c31a57d-istiod-ca-cert\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc\" (UID: \"f506e56e-854d-4a96-a255-f04c7c31a57d\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"
Oct 30 21:59:29.765455 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.738679    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f506e56e-854d-4a96-a255-f04c7c31a57d-workload-certs\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc\" (UID: \"f506e56e-854d-4a96-a255-f04c7c31a57d\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"
Oct 30 21:59:29.765455 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.738713    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f506e56e-854d-4a96-a255-f04c7c31a57d-istio-envoy\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc\" (UID: \"f506e56e-854d-4a96-a255-f04c7c31a57d\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"
Oct 30 21:59:29.765455 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.738788    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f506e56e-854d-4a96-a255-f04c7c31a57d-istio-data\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc\" (UID: \"f506e56e-854d-4a96-a255-f04c7c31a57d\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"
Oct 30 21:59:29.765455 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.738853    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f506e56e-854d-4a96-a255-f04c7c31a57d-istio-podinfo\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc\" (UID: \"f506e56e-854d-4a96-a255-f04c7c31a57d\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"
Oct 30 21:59:29.765455 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.738874    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f506e56e-854d-4a96-a255-f04c7c31a57d-istio-token\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc\" (UID: \"f506e56e-854d-4a96-a255-f04c7c31a57d\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"
Oct 30 21:59:29.765455 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.738954    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f506e56e-854d-4a96-a255-f04c7c31a57d-workload-socket\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc\" (UID: \"f506e56e-854d-4a96-a255-f04c7c31a57d\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"
Oct 30 21:59:29.766913 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.766874    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f506e56e-854d-4a96-a255-f04c7c31a57d-workload-socket\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc\" (UID: \"f506e56e-854d-4a96-a255-f04c7c31a57d\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"
Oct 30 21:59:29.767459 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.767434    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f506e56e-854d-4a96-a255-f04c7c31a57d-workload-certs\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc\" (UID: \"f506e56e-854d-4a96-a255-f04c7c31a57d\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"
Oct 30 21:59:29.767884 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.767863    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f506e56e-854d-4a96-a255-f04c7c31a57d-credential-socket\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc\" (UID: \"f506e56e-854d-4a96-a255-f04c7c31a57d\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"
Oct 30 21:59:29.801277 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.768906    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f506e56e-854d-4a96-a255-f04c7c31a57d-istiod-ca-cert\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc\" (UID: \"f506e56e-854d-4a96-a255-f04c7c31a57d\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"
Oct 30 21:59:29.803661 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.803629    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f506e56e-854d-4a96-a255-f04c7c31a57d-istio-data\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc\" (UID: \"f506e56e-854d-4a96-a255-f04c7c31a57d\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"
Oct 30 21:59:29.871312 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.805340    4871 patch_prober.go:29] interesting pod/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 21:59:29.871312 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.865939    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2" podUID="85541021-4964-4ce1-b471-9df326efc190" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 21:59:29.915734 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.877312    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bspwc\" (UniqueName: \"kubernetes.io/projected/47d9d464-c7c2-4980-ba4d-e3320373ed88-kube-api-access-bspwc\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh\" (UID: \"47d9d464-c7c2-4980-ba4d-e3320373ed88\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"
Oct 30 21:59:29.915734 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.905066    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5whcd\" (UniqueName: \"kubernetes.io/projected/f506e56e-854d-4a96-a255-f04c7c31a57d-kube-api-access-5whcd\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc\" (UID: \"f506e56e-854d-4a96-a255-f04c7c31a57d\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"
Oct 30 21:59:29.915734 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.909688    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f506e56e-854d-4a96-a255-f04c7c31a57d-istio-token\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc\" (UID: \"f506e56e-854d-4a96-a255-f04c7c31a57d\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"
Oct 30 21:59:29.935649 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:29.935607    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f506e56e-854d-4a96-a255-f04c7c31a57d-istio-envoy\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc\" (UID: \"f506e56e-854d-4a96-a255-f04c7c31a57d\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"
Oct 30 21:59:30.047681 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:30.047601    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47d9d464-c7c2-4980-ba4d-e3320373ed88-istio-envoy\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh\" (UID: \"47d9d464-c7c2-4980-ba4d-e3320373ed88\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"
Oct 30 21:59:30.048574 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:30.048547    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47d9d464-c7c2-4980-ba4d-e3320373ed88-istio-podinfo\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh\" (UID: \"47d9d464-c7c2-4980-ba4d-e3320373ed88\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"
Oct 30 21:59:30.078272 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:30.078154    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47d9d464-c7c2-4980-ba4d-e3320373ed88-istio-token\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh\" (UID: \"47d9d464-c7c2-4980-ba4d-e3320373ed88\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"
Oct 30 21:59:30.092955 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:30.092917    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f506e56e-854d-4a96-a255-f04c7c31a57d-istio-podinfo\") pod \"e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc\" (UID: \"f506e56e-854d-4a96-a255-f04c7c31a57d\") " pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"
Oct 30 21:59:30.203701 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:30.203610    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"
Oct 30 21:59:30.413260 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:30.408501    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"
Oct 30 21:59:30.608543 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:59:30.608501    4871 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-00e4c0ba7157: secret "agc-cred-00e4c0ba7157" not found
Oct 30 21:59:30.608543 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:59:30.608542    4871 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-3d0f8113fe57: secret "agc-cred-3d0f8113fe57" not found
Oct 30 21:59:30.608543 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:59:30.608553    4871 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-7653478ffe43: secret "agc-cred-7653478ffe43" not found
Oct 30 21:59:30.609169 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:59:30.608567    4871 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-78677bf7-fm4j7: [secret "agc-cred-00e4c0ba7157" not found, secret "agc-cred-3d0f8113fe57" not found, secret "agc-cred-7653478ffe43" not found]
Oct 30 21:59:30.609169 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:59:30.608627    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/33d005ac-b847-4fd5-af8f-5b0162b56610-provider-credentials podName:33d005ac-b847-4fd5-af8f-5b0162b56610 nodeName:}" failed. No retries permitted until 2026-10-30 21:59:38.608607991 +0000 UTC m=+1320.786096586 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/33d005ac-b847-4fd5-af8f-5b0162b56610-provider-credentials") pod "payload-processing-external-model-78677bf7-fm4j7" (UID: "33d005ac-b847-4fd5-af8f-5b0162b56610") : [secret "agc-cred-00e4c0ba7157" not found, secret "agc-cred-3d0f8113fe57" not found, secret "agc-cred-7653478ffe43" not found]
Oct 30 21:59:30.639265 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:30.634025    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 30 21:59:30.639265 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:30.634088    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt" podUID="52b9b4dc-149f-4650-81f7-7f0234d9c6d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 30 21:59:30.801361 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:30.782629    4871 patch_prober.go:29] interesting pod/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 21:59:30.801361 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:30.782695    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2" podUID="85541021-4964-4ce1-b471-9df326efc190" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 21:59:31.575272 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:31.542913    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-78677bf7-fm4j7" podUID="33d005ac-b847-4fd5-af8f-5b0162b56610" containerName="payload-processing" containerID="cri-o://a6d16e53d08a4a6d60593abc608c1dd94d3331a6d1261e268fcc64e692484c96" gracePeriod=25
Oct 30 21:59:31.608263 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:31.567278    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-866b9fd699-kvmpl" podUID="7c3f8565-5b8f-4709-bacb-773e5abc4659" containerName="payload-processing" containerID="cri-o://dd1c30fbe58d957e68c1f3ea86a425c2f56141eee3607dda8d4d1a0f3380ff37" gracePeriod=25
Oct 30 21:59:31.862714 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:31.637579    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:59:31.862714 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:31.646599    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:59:31.862714 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:31.646662    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:59:31.862714 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:31.707552    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"]
Oct 30 21:59:31.908183 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:31.878771    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 30 21:59:31.908183 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:31.878934    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt" podUID="52b9b4dc-149f-4650-81f7-7f0234d9c6d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 30 21:59:31.908183 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:31.884846    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"]
Oct 30 21:59:31.908183 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:31.885557    4871 patch_prober.go:29] interesting pod/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 21:59:31.908183 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:31.885605    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2" podUID="85541021-4964-4ce1-b471-9df326efc190" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 21:59:32.147614 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:32.054678    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:32.292259 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:32.289604    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:59:32.292259 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:32.290335    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:59:32.292259 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:32.291005    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:59:32.694394 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:32.612776    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 30 21:59:32.694394 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:32.612849    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt" podUID="52b9b4dc-149f-4650-81f7-7f0234d9c6d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 30 21:59:32.849073 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:32.849030    4871 patch_prober.go:29] interesting pod/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 21:59:32.876276 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:32.849337    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2" podUID="85541021-4964-4ce1-b471-9df326efc190" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 21:59:32.981351 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:32.973412    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8" podUID="84329f90-dc35-41dc-b593-12c556e80f73" containerName="main" probeResult="failure" output="Get \"https://10.133.0.83:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:59:33.066069 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:33.066012    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc" event={"ID":"f506e56e-854d-4a96-a255-f04c7c31a57d","Type":"ContainerStarted","Data":"66602091e4511816fafc75a5fb9e9923c80c076440f01b97cc7fa0047afe885a"}
Oct 30 21:59:33.691921 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:33.688455    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 30 21:59:33.691921 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:33.688526    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt" podUID="52b9b4dc-149f-4650-81f7-7f0234d9c6d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 30 21:59:33.797266 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:33.796174    4871 patch_prober.go:29] interesting pod/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 21:59:33.797266 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:33.796255    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2" podUID="85541021-4964-4ce1-b471-9df326efc190" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 21:59:34.326763 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:34.326712    4871 generic.go:356] "Generic (PLEG): container finished" podID="7c3f8565-5b8f-4709-bacb-773e5abc4659" containerID="dd1c30fbe58d957e68c1f3ea86a425c2f56141eee3607dda8d4d1a0f3380ff37" exitCode=0
Oct 30 21:59:34.327483 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:34.327329    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-866b9fd699-kvmpl" event={"ID":"7c3f8565-5b8f-4709-bacb-773e5abc4659","Type":"ContainerDied","Data":"dd1c30fbe58d957e68c1f3ea86a425c2f56141eee3607dda8d4d1a0f3380ff37"}
Oct 30 21:59:34.436679 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:34.436629    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh" event={"ID":"47d9d464-c7c2-4980-ba4d-e3320373ed88","Type":"ContainerStarted","Data":"36d126e8c3b1236bead43b361107f7a56b366e05d4ddb1072f4322ebf2fcba1c"}
Oct 30 21:59:34.437403 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:34.437353    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.94:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:59:34.437571 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:34.437419    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs" podUID="78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.94:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:59:34.462930 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:34.462653    4871 generic.go:356] "Generic (PLEG): container finished" podID="33d005ac-b847-4fd5-af8f-5b0162b56610" containerID="a6d16e53d08a4a6d60593abc608c1dd94d3331a6d1261e268fcc64e692484c96" exitCode=0
Oct 30 21:59:34.462930 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:34.462804    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-78677bf7-fm4j7" event={"ID":"33d005ac-b847-4fd5-af8f-5b0162b56610","Type":"ContainerDied","Data":"a6d16e53d08a4a6d60593abc608c1dd94d3331a6d1261e268fcc64e692484c96"}
Oct 30 21:59:34.621280 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:34.621221    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 30 21:59:34.621655 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:34.621624    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt" podUID="52b9b4dc-149f-4650-81f7-7f0234d9c6d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 30 21:59:34.900322 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:34.899886    4871 patch_prober.go:29] interesting pod/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 21:59:34.900322 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:34.899949    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2" podUID="85541021-4964-4ce1-b471-9df326efc190" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 21:59:35.623010 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:35.622974    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 30 21:59:35.640457 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:35.640348    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt" podUID="52b9b4dc-149f-4650-81f7-7f0234d9c6d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 30 21:59:35.642077 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:35.641715    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:35.742169 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:35.735141    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh" event={"ID":"47d9d464-c7c2-4980-ba4d-e3320373ed88","Type":"ContainerStarted","Data":"88cce796c963ab6a228d9846d6b5114217db53e246ee8de0a3c3871a35e3cfbc"}
Oct 30 21:59:35.824312 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:35.813404    4871 patch_prober.go:29] interesting pod/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 21:59:35.824312 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:35.813482    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2" podUID="85541021-4964-4ce1-b471-9df326efc190" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 21:59:36.158264 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:36.125281    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc" event={"ID":"f506e56e-854d-4a96-a255-f04c7c31a57d","Type":"ContainerStarted","Data":"b1feeecb95544ceb0f7d8bc42c0d429ecc95e01a57ad527803c500c3d983b47a"}
Oct 30 21:59:36.215745 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:36.208357    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh" podStartSLOduration=7.208334871 podStartE2EDuration="7.208334871s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:59:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:59:35.931097752 +0000 UTC m=+1318.108586369" watchObservedRunningTime="2026-10-30 21:59:36.208334871 +0000 UTC m=+1318.385823488"
Oct 30 21:59:36.215745 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:36.214162    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"
Oct 30 21:59:36.218447 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:36.218411    4871 patch_prober.go:29] interesting pod/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused" start-of-body=
Oct 30 21:59:36.218684 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:36.218658    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc" podUID="f506e56e-854d-4a96-a255-f04c7c31a57d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused"
Oct 30 21:59:36.350265 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:36.347048    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc" podStartSLOduration=7.347024616 podStartE2EDuration="7.347024616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:59:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:59:36.108953149 +0000 UTC m=+1318.286441766" watchObservedRunningTime="2026-10-30 21:59:36.347024616 +0000 UTC m=+1318.524513233"
Oct 30 21:59:36.420360 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:36.420318    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"
Oct 30 21:59:36.467181 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:36.467140    4871 patch_prober.go:29] interesting pod/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused" start-of-body=
Oct 30 21:59:36.467511 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:36.467476    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh" podUID="47d9d464-c7c2-4980-ba4d-e3320373ed88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused"
Oct 30 21:59:36.553261 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:36.546062    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-866b9fd699-kvmpl"
Oct 30 21:59:36.560486 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:36.560450    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-78677bf7-fm4j7"
Oct 30 21:59:36.643408 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:36.643368    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 30 21:59:36.644010 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:36.643976    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt" podUID="52b9b4dc-149f-4650-81f7-7f0234d9c6d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 30 21:59:36.671314 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:59:36.668561    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 21:59:36.719721 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:36.719675    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7c3f8565-5b8f-4709-bacb-773e5abc4659-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c3f8565-5b8f-4709-bacb-773e5abc4659-plugins-config-volume\") pod \"7c3f8565-5b8f-4709-bacb-773e5abc4659\" (UID: \"7c3f8565-5b8f-4709-bacb-773e5abc4659\") "
Oct 30 21:59:36.720022 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:36.719998    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/33d005ac-b847-4fd5-af8f-5b0162b56610-provider-credentials\" (UniqueName: \"kubernetes.io/projected/33d005ac-b847-4fd5-af8f-5b0162b56610-provider-credentials\") pod \"33d005ac-b847-4fd5-af8f-5b0162b56610\" (UID: \"33d005ac-b847-4fd5-af8f-5b0162b56610\") "
Oct 30 21:59:36.720179 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:36.720159    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/33d005ac-b847-4fd5-af8f-5b0162b56610-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/33d005ac-b847-4fd5-af8f-5b0162b56610-routing-overlay-volume\") pod \"33d005ac-b847-4fd5-af8f-5b0162b56610\" (UID: \"33d005ac-b847-4fd5-af8f-5b0162b56610\") "
Oct 30 21:59:36.720379 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:36.720361    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7c3f8565-5b8f-4709-bacb-773e5abc4659-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/7c3f8565-5b8f-4709-bacb-773e5abc4659-routing-overlay-volume\") pod \"7c3f8565-5b8f-4709-bacb-773e5abc4659\" (UID: \"7c3f8565-5b8f-4709-bacb-773e5abc4659\") "
Oct 30 21:59:36.720524 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:36.720509    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/33d005ac-b847-4fd5-af8f-5b0162b56610-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33d005ac-b847-4fd5-af8f-5b0162b56610-plugins-config-volume\") pod \"33d005ac-b847-4fd5-af8f-5b0162b56610\" (UID: \"33d005ac-b847-4fd5-af8f-5b0162b56610\") "
Oct 30 21:59:36.757263 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:36.720666    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7c3f8565-5b8f-4709-bacb-773e5abc4659-kube-api-access-c54r9\" (UniqueName: \"kubernetes.io/projected/7c3f8565-5b8f-4709-bacb-773e5abc4659-kube-api-access-c54r9\") pod \"7c3f8565-5b8f-4709-bacb-773e5abc4659\" (UID: \"7c3f8565-5b8f-4709-bacb-773e5abc4659\") "
Oct 30 21:59:36.757673 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:36.757638    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/33d005ac-b847-4fd5-af8f-5b0162b56610-kube-api-access-jrxl4\" (UniqueName: \"kubernetes.io/projected/33d005ac-b847-4fd5-af8f-5b0162b56610-kube-api-access-jrxl4\") pod \"33d005ac-b847-4fd5-af8f-5b0162b56610\" (UID: \"33d005ac-b847-4fd5-af8f-5b0162b56610\") "
Oct 30 21:59:36.757919 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:36.757899    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7c3f8565-5b8f-4709-bacb-773e5abc4659-provider-credentials\" (UniqueName: \"kubernetes.io/projected/7c3f8565-5b8f-4709-bacb-773e5abc4659-provider-credentials\") pod \"7c3f8565-5b8f-4709-bacb-773e5abc4659\" (UID: \"7c3f8565-5b8f-4709-bacb-773e5abc4659\") "
Oct 30 21:59:36.760773 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:36.759801    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c3f8565-5b8f-4709-bacb-773e5abc4659-routing-overlay-volume" pod "7c3f8565-5b8f-4709-bacb-773e5abc4659" (UID: "7c3f8565-5b8f-4709-bacb-773e5abc4659"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:59:36.760773 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:36.760268    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c3f8565-5b8f-4709-bacb-773e5abc4659-plugins-config-volume" pod "7c3f8565-5b8f-4709-bacb-773e5abc4659" (UID: "7c3f8565-5b8f-4709-bacb-773e5abc4659"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:59:36.760773 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:36.760736    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33d005ac-b847-4fd5-af8f-5b0162b56610-plugins-config-volume" pod "33d005ac-b847-4fd5-af8f-5b0162b56610" (UID: "33d005ac-b847-4fd5-af8f-5b0162b56610"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:59:36.761528 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:36.761482    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33d005ac-b847-4fd5-af8f-5b0162b56610-routing-overlay-volume" pod "33d005ac-b847-4fd5-af8f-5b0162b56610" (UID: "33d005ac-b847-4fd5-af8f-5b0162b56610"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:59:36.801388 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:36.801348    4871 patch_prober.go:29] interesting pod/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 21:59:36.801710 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:36.801673    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2" podUID="85541021-4964-4ce1-b471-9df326efc190" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 21:59:36.874270 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:36.861300    4871 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c3f8565-5b8f-4709-bacb-773e5abc4659-plugins-config-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:36.874270 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:36.861338    4871 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/33d005ac-b847-4fd5-af8f-5b0162b56610-routing-overlay-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:36.874270 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:36.861355    4871 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/7c3f8565-5b8f-4709-bacb-773e5abc4659-routing-overlay-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:36.874270 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:36.861369    4871 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33d005ac-b847-4fd5-af8f-5b0162b56610-plugins-config-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:36.907971 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:36.907907    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c3f8565-5b8f-4709-bacb-773e5abc4659-kube-api-access-c54r9" pod "7c3f8565-5b8f-4709-bacb-773e5abc4659" (UID: "7c3f8565-5b8f-4709-bacb-773e5abc4659"). InnerVolumeSpecName "kube-api-access-c54r9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:59:36.934329 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:36.930418    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c3f8565-5b8f-4709-bacb-773e5abc4659-provider-credentials" pod "7c3f8565-5b8f-4709-bacb-773e5abc4659" (UID: "7c3f8565-5b8f-4709-bacb-773e5abc4659"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:59:36.958471 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:36.958421    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33d005ac-b847-4fd5-af8f-5b0162b56610-kube-api-access-jrxl4" pod "33d005ac-b847-4fd5-af8f-5b0162b56610" (UID: "33d005ac-b847-4fd5-af8f-5b0162b56610"). InnerVolumeSpecName "kube-api-access-jrxl4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:59:36.974629 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:36.974527    4871 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/7c3f8565-5b8f-4709-bacb-773e5abc4659-provider-credentials\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:36.974629 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:36.974571    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c54r9\" (UniqueName: \"kubernetes.io/projected/7c3f8565-5b8f-4709-bacb-773e5abc4659-kube-api-access-c54r9\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:36.974629 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:36.974588    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrxl4\" (UniqueName: \"kubernetes.io/projected/33d005ac-b847-4fd5-af8f-5b0162b56610-kube-api-access-jrxl4\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:36.978468 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:36.978399    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33d005ac-b847-4fd5-af8f-5b0162b56610-provider-credentials" pod "33d005ac-b847-4fd5-af8f-5b0162b56610" (UID: "33d005ac-b847-4fd5-af8f-5b0162b56610"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:59:37.320323 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:37.314943    4871 patch_prober.go:29] interesting pod/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused" start-of-body=
Oct 30 21:59:37.320323 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:37.315006    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc" podUID="f506e56e-854d-4a96-a255-f04c7c31a57d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused"
Oct 30 21:59:37.320323 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:37.318640    4871 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/33d005ac-b847-4fd5-af8f-5b0162b56610-provider-credentials\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:37.324261 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:37.321044    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-866b9fd699-kvmpl" podUID="7c3f8565-5b8f-4709-bacb-773e5abc4659" containerName="payload-processing" probeResult="failure" output="timeout: failed to connect service \"10.133.0.97:50052\" within 1s: context deadline exceeded"
Oct 30 21:59:37.345498 ip-10-0-139-46 kubenswrapper[4871]: W1030 21:59:37.345445    4871 logging.go:55] [core] [Channel #1595 SubChannel #1596]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.97:50052", ServerName: "10.133.0.97:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.97:50052: operation was canceled"
Oct 30 21:59:37.371218 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:37.350608    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-78677bf7-fm4j7" event={"ID":"33d005ac-b847-4fd5-af8f-5b0162b56610","Type":"ContainerDied","Data":"c9123c8e980089875a145452c1676f57abed23489b99c13fb5460fb5cbf5861c"}
Oct 30 21:59:37.371218 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:37.350686    4871 scope.go:122] "RemoveContainer" containerID="a6d16e53d08a4a6d60593abc608c1dd94d3331a6d1261e268fcc64e692484c96"
Oct 30 21:59:37.371218 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:37.350862    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-78677bf7-fm4j7"
Oct 30 21:59:37.574319 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:37.558292    4871 patch_prober.go:29] interesting pod/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused" start-of-body=
Oct 30 21:59:37.574319 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:37.558370    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh" podUID="47d9d464-c7c2-4980-ba4d-e3320373ed88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused"
Oct 30 21:59:37.574319 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:37.565269    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-866b9fd699-kvmpl"
Oct 30 21:59:37.576573 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:37.576524    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-866b9fd699-kvmpl" event={"ID":"7c3f8565-5b8f-4709-bacb-773e5abc4659","Type":"ContainerDied","Data":"fcb17ea4d6d400b674fd54a5935a017a5423fc644b294bc146758097acfc860d"}
Oct 30 21:59:37.738538 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:37.738497    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 30 21:59:37.809414 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:37.809370    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt" podUID="52b9b4dc-149f-4650-81f7-7f0234d9c6d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 30 21:59:38.101269 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:38.100663    4871 scope.go:122] "RemoveContainer" containerID="dd1c30fbe58d957e68c1f3ea86a425c2f56141eee3607dda8d4d1a0f3380ff37"
Oct 30 21:59:38.198262 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:38.155174    4871 patch_prober.go:29] interesting pod/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 21:59:38.198262 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:38.155262    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2" podUID="85541021-4964-4ce1-b471-9df326efc190" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 21:59:38.562888 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:38.498416    4871 patch_prober.go:29] interesting pod/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused" start-of-body=
Oct 30 21:59:38.562888 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:38.498474    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc" podUID="f506e56e-854d-4a96-a255-f04c7c31a57d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused"
Oct 30 21:59:38.789261 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:38.786421    4871 patch_prober.go:29] interesting pod/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 21:59:38.789261 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:38.786573    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2" podUID="85541021-4964-4ce1-b471-9df326efc190" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 21:59:38.789261 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:38.786931    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 30 21:59:38.789261 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:38.787072    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt" podUID="52b9b4dc-149f-4650-81f7-7f0234d9c6d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 30 21:59:38.794600 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:38.794570    4871 patch_prober.go:29] interesting pod/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused" start-of-body=
Oct 30 21:59:38.794832 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:38.794808    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh" podUID="47d9d464-c7c2-4980-ba4d-e3320373ed88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused"
Oct 30 21:59:38.994111 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:38.994066    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-866b9fd699-kvmpl"]
Oct 30 21:59:38.994397 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:38.994376    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-866b9fd699-kvmpl"]
Oct 30 21:59:39.056363 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:39.056263    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:39.164347 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:39.164295    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-78677bf7-fm4j7"]
Oct 30 21:59:39.213648 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:39.213606    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-78677bf7-fm4j7"]
Oct 30 21:59:39.214008 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:39.213923    4871 patch_prober.go:29] interesting pod/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused" start-of-body=
Oct 30 21:59:39.214190 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:39.214165    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc" podUID="f506e56e-854d-4a96-a255-f04c7c31a57d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused"
Oct 30 21:59:39.426786 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:39.426735    4871 patch_prober.go:29] interesting pod/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused" start-of-body=
Oct 30 21:59:39.427070 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:39.427041    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh" podUID="47d9d464-c7c2-4980-ba4d-e3320373ed88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused"
Oct 30 21:59:39.613736 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:39.613691    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 30 21:59:39.614014 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:39.613983    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt" podUID="52b9b4dc-149f-4650-81f7-7f0234d9c6d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 30 21:59:39.794318 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:39.789920    4871 patch_prober.go:29] interesting pod/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 21:59:39.794318 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:39.789992    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2" podUID="85541021-4964-4ce1-b471-9df326efc190" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 21:59:40.157263 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:40.137962    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="33d005ac-b847-4fd5-af8f-5b0162b56610" path="/var/lib/kubelet/pods/33d005ac-b847-4fd5-af8f-5b0162b56610/volumes"
Oct 30 21:59:40.159263 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:40.158768    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7c3f8565-5b8f-4709-bacb-773e5abc4659" path="/var/lib/kubelet/pods/7c3f8565-5b8f-4709-bacb-773e5abc4659/volumes"
Oct 30 21:59:40.207675 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:40.207631    4871 patch_prober.go:29] interesting pod/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused" start-of-body=
Oct 30 21:59:40.207934 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:40.207900    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc" podUID="f506e56e-854d-4a96-a255-f04c7c31a57d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused"
Oct 30 21:59:40.450320 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:40.450093    4871 patch_prober.go:29] interesting pod/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused" start-of-body=
Oct 30 21:59:40.450320 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:40.450156    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh" podUID="47d9d464-c7c2-4980-ba4d-e3320373ed88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused"
Oct 30 21:59:40.727955 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:40.619437    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 30 21:59:40.753428 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:40.753374    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt" podUID="52b9b4dc-149f-4650-81f7-7f0234d9c6d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 30 21:59:40.786313 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:40.786195    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"
Oct 30 21:59:41.205786 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:41.205747    4871 patch_prober.go:29] interesting pod/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused" start-of-body=
Oct 30 21:59:41.206439 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:41.206393    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc" podUID="f506e56e-854d-4a96-a255-f04c7c31a57d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused"
Oct 30 21:59:41.412683 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:41.412636    4871 patch_prober.go:29] interesting pod/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused" start-of-body=
Oct 30 21:59:41.413019 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:41.412985    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh" podUID="47d9d464-c7c2-4980-ba4d-e3320373ed88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused"
Oct 30 21:59:41.614326 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:41.611758    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 30 21:59:41.614326 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:41.611827    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt" podUID="52b9b4dc-149f-4650-81f7-7f0234d9c6d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 30 21:59:41.794411 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:41.791768    4871 patch_prober.go:29] interesting pod/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:59:41.794411 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:41.791837    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2" podUID="85541021-4964-4ce1-b471-9df326efc190" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:59:42.208442 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:42.204748    4871 patch_prober.go:29] interesting pod/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused" start-of-body=
Oct 30 21:59:42.208442 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:42.204827    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc" podUID="f506e56e-854d-4a96-a255-f04c7c31a57d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused"
Oct 30 21:59:42.412855 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:42.412763    4871 patch_prober.go:29] interesting pod/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused" start-of-body=
Oct 30 21:59:42.413131 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:42.413086    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh" podUID="47d9d464-c7c2-4980-ba4d-e3320373ed88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused"
Oct 30 21:59:42.618293 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:42.617305    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"
Oct 30 21:59:42.842481 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:42.842363    4871 patch_prober.go:29] interesting pod/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:59:42.842776 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:42.842719    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2" podUID="85541021-4964-4ce1-b471-9df326efc190" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:59:42.935718 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:42.935673    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"
Oct 30 21:59:43.214264 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:43.207740    4871 patch_prober.go:29] interesting pod/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused" start-of-body=
Oct 30 21:59:43.214264 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:43.207822    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc" podUID="f506e56e-854d-4a96-a255-f04c7c31a57d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused"
Oct 30 21:59:43.292929 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:43.292899    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"
Oct 30 21:59:43.413498 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:43.413455    4871 patch_prober.go:29] interesting pod/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused" start-of-body=
Oct 30 21:59:43.413749 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:43.413724    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh" podUID="47d9d464-c7c2-4980-ba4d-e3320373ed88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused"
Oct 30 21:59:43.546650 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:43.546567    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"
Oct 30 21:59:43.549688 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:43.549653    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"
Oct 30 21:59:44.204642 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:44.204601    4871 patch_prober.go:29] interesting pod/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused" start-of-body=
Oct 30 21:59:44.204834 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:44.204675    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc" podUID="f506e56e-854d-4a96-a255-f04c7c31a57d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused"
Oct 30 21:59:44.440755 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:44.440664    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:44.892264 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:44.892202    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"
Oct 30 21:59:44.893969 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:44.893946    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"
Oct 30 21:59:44.897447 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:44.897417    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"
Oct 30 21:59:45.204586 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:45.204475    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"
Oct 30 21:59:45.211524 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:45.211493    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"
Oct 30 21:59:45.307894 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:45.307864    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"
Oct 30 21:59:46.099002 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:59:46.098965    4871 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33d005ac_b847_4fd5_af8f_5b0162b56610.slice\": RecentStats: unable to find data in memory cache]"
Oct 30 21:59:47.848072 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:47.848032    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:48.032007 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:48.031974    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:49.032378 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:49.032320    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:49.249120 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:49.249085    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:49.448036 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:49.447981    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:49.713230 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:59:49.713136    4871 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33d005ac_b847_4fd5_af8f_5b0162b56610.slice\": RecentStats: unable to find data in memory cache]"
Oct 30 21:59:53.894263 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:53.887130    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:56.182541 ip-10-0-139-46 kubenswrapper[4871]: E1030 21:59:56.182495    4871 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33d005ac_b847_4fd5_af8f_5b0162b56610.slice\": RecentStats: unable to find data in memory cache]"
Oct 30 21:59:57.968855 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:57.968687    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:58.146262 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:58.142941    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:58.864385 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:58.856422    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8" podUID="84329f90-dc35-41dc-b593-12c556e80f73" containerName="main" probeResult="failure" output="Get \"https://10.133.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:59:59.872598 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:59.872547    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8" podUID="84329f90-dc35-41dc-b593-12c556e80f73" containerName="main" probeResult="failure" output="Get \"https://10.133.0.83:8000/ready\": context deadline exceeded"
Oct 30 21:59:59.888023 ip-10-0-139-46 kubenswrapper[4871]: I1030 21:59:59.887987    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8"
Oct 30 22:00:00.230383 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:00.230292    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8"
Oct 30 22:00:00.553325 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:00.475298    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889960-ng4ws"]
Oct 30 22:00:00.553325 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:00.476255    4871 cpu_manager.go:414] "containerMap: removing container" podUID="7c3f8565-5b8f-4709-bacb-773e5abc4659" containerName="payload-processing"
Oct 30 22:00:00.553325 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:00.476280    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7c3f8565-5b8f-4709-bacb-773e5abc4659" containerName="payload-processing"
Oct 30 22:00:00.553325 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:00.476298    4871 cpu_manager.go:414] "containerMap: removing container" podUID="33d005ac-b847-4fd5-af8f-5b0162b56610" containerName="payload-processing"
Oct 30 22:00:00.553325 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:00.476305    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="33d005ac-b847-4fd5-af8f-5b0162b56610" containerName="payload-processing"
Oct 30 22:00:00.553325 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:00.476518    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="7c3f8565-5b8f-4709-bacb-773e5abc4659" containerName="payload-processing"
Oct 30 22:00:00.553325 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:00.476534    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="33d005ac-b847-4fd5-af8f-5b0162b56610" containerName="payload-processing"
Oct 30 22:00:00.553325 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:00.480782    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889960-ng4ws"
Oct 30 22:00:00.553325 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:00.526353    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:00.579676 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:00.579639    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mlvtf\""
Oct 30 22:00:00.623269 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:00.617071    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed194f68-2c40-49ae-b9a4-0680e645ecec-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-26743a-29889960-ng4ws\" (UID: \"ed194f68-2c40-49ae-b9a4-0680e645ecec\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889960-ng4ws"
Oct 30 22:00:00.636469 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:00.636426    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6p4q5\" (UniqueName: \"kubernetes.io/projected/ed194f68-2c40-49ae-b9a4-0680e645ecec-kube-api-access-6p4q5\") pod \"maas-api-key-cleanup-e2e-worker-w4-26743a-29889960-ng4ws\" (UID: \"ed194f68-2c40-49ae-b9a4-0680e645ecec\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889960-ng4ws"
Oct 30 22:00:00.697367 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:00.697325    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9a5623-29889960-t8mjd"]
Oct 30 22:00:00.724730 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:00.724698    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9a5623-29889960-t8mjd"
Oct 30 22:00:00.743739 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:00.742156    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcsg5\" (UniqueName: \"kubernetes.io/projected/4b6736b8-3237-4e98-8b56-132f67660bd1-kube-api-access-lcsg5\") pod \"maas-api-key-cleanup-e2e-worker-w0-9a5623-29889960-t8mjd\" (UID: \"4b6736b8-3237-4e98-8b56-132f67660bd1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9a5623-29889960-t8mjd"
Oct 30 22:00:00.743739 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:00.742321    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed194f68-2c40-49ae-b9a4-0680e645ecec-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-26743a-29889960-ng4ws\" (UID: \"ed194f68-2c40-49ae-b9a4-0680e645ecec\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889960-ng4ws"
Oct 30 22:00:00.743739 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:00.742399    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6p4q5\" (UniqueName: \"kubernetes.io/projected/ed194f68-2c40-49ae-b9a4-0680e645ecec-kube-api-access-6p4q5\") pod \"maas-api-key-cleanup-e2e-worker-w4-26743a-29889960-ng4ws\" (UID: \"ed194f68-2c40-49ae-b9a4-0680e645ecec\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889960-ng4ws"
Oct 30 22:00:00.743739 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:00.742478    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b6736b8-3237-4e98-8b56-132f67660bd1-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-9a5623-29889960-t8mjd\" (UID: \"4b6736b8-3237-4e98-8b56-132f67660bd1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9a5623-29889960-t8mjd"
Oct 30 22:00:00.743739 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:00.743406    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed194f68-2c40-49ae-b9a4-0680e645ecec-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-26743a-29889960-ng4ws\" (UID: \"ed194f68-2c40-49ae-b9a4-0680e645ecec\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889960-ng4ws"
Oct 30 22:00:00.766718 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:00.764956    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889960-ng4ws"]
Oct 30 22:00:00.876933 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:00.848094    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b6736b8-3237-4e98-8b56-132f67660bd1-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-9a5623-29889960-t8mjd\" (UID: \"4b6736b8-3237-4e98-8b56-132f67660bd1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9a5623-29889960-t8mjd"
Oct 30 22:00:00.876933 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:00.848228    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcsg5\" (UniqueName: \"kubernetes.io/projected/4b6736b8-3237-4e98-8b56-132f67660bd1-kube-api-access-lcsg5\") pod \"maas-api-key-cleanup-e2e-worker-w0-9a5623-29889960-t8mjd\" (UID: \"4b6736b8-3237-4e98-8b56-132f67660bd1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9a5623-29889960-t8mjd"
Oct 30 22:00:00.876933 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:00.849562    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b6736b8-3237-4e98-8b56-132f67660bd1-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-9a5623-29889960-t8mjd\" (UID: \"4b6736b8-3237-4e98-8b56-132f67660bd1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9a5623-29889960-t8mjd"
Oct 30 22:00:00.946373 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:00.946214    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9a5623-29889960-t8mjd"]
Oct 30 22:00:00.993205 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:00.991969    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6p4q5\" (UniqueName: \"kubernetes.io/projected/ed194f68-2c40-49ae-b9a4-0680e645ecec-kube-api-access-6p4q5\") pod \"maas-api-key-cleanup-e2e-worker-w4-26743a-29889960-ng4ws\" (UID: \"ed194f68-2c40-49ae-b9a4-0680e645ecec\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889960-ng4ws"
Oct 30 22:00:01.021261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:01.018272    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcsg5\" (UniqueName: \"kubernetes.io/projected/4b6736b8-3237-4e98-8b56-132f67660bd1-kube-api-access-lcsg5\") pod \"maas-api-key-cleanup-e2e-worker-w0-9a5623-29889960-t8mjd\" (UID: \"4b6736b8-3237-4e98-8b56-132f67660bd1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9a5623-29889960-t8mjd"
Oct 30 22:00:01.130320 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:01.129478    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889960-ng4ws"
Oct 30 22:00:01.307267 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:01.307195    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9a5623-29889960-t8mjd"
Oct 30 22:00:02.128811 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:02.128779    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889960-ng4ws"]
Oct 30 22:00:02.317505 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:02.317329    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889960-ng4ws" event={"ID":"ed194f68-2c40-49ae-b9a4-0680e645ecec","Type":"ContainerStarted","Data":"1a861436245a062a45616327f9fdaa93e2f7fc3483c98252750df1f835dec767"}
Oct 30 22:00:02.376509 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:02.376462    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9a5623-29889960-t8mjd"]
Oct 30 22:00:02.407272 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:00:02.406905    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b6736b8_3237_4e98_8b56_132f67660bd1.slice/crio-0196fc00412aedfe9199aeaaec9bd76f8bdb19d2c746aecf3bf139677f972299 WatchSource:0}: Error finding container 0196fc00412aedfe9199aeaaec9bd76f8bdb19d2c746aecf3bf139677f972299: Status 404 returned error can't find the container with id 0196fc00412aedfe9199aeaaec9bd76f8bdb19d2c746aecf3bf139677f972299
Oct 30 22:00:02.541005 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:02.540955    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj"]
Oct 30 22:00:02.547755 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:02.547729    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj"
Oct 30 22:00:02.577640 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:02.577000    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93c5282f-81b6-4518-b673-68f527fdfd04-home\") pod \"distinct-w4-26743a-kserve-85985748cd-vl2fj\" (UID: \"93c5282f-81b6-4518-b673-68f527fdfd04\") " pod="e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj"
Oct 30 22:00:02.577640 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:02.577053    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93c5282f-81b6-4518-b673-68f527fdfd04-tls-certs\") pod \"distinct-w4-26743a-kserve-85985748cd-vl2fj\" (UID: \"93c5282f-81b6-4518-b673-68f527fdfd04\") " pod="e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj"
Oct 30 22:00:02.577640 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:02.577115    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93c5282f-81b6-4518-b673-68f527fdfd04-model-cache\") pod \"distinct-w4-26743a-kserve-85985748cd-vl2fj\" (UID: \"93c5282f-81b6-4518-b673-68f527fdfd04\") " pod="e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj"
Oct 30 22:00:02.577640 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:02.577146    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93c5282f-81b6-4518-b673-68f527fdfd04-dshm\") pod \"distinct-w4-26743a-kserve-85985748cd-vl2fj\" (UID: \"93c5282f-81b6-4518-b673-68f527fdfd04\") " pod="e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj"
Oct 30 22:00:02.577640 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:02.577212    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93c5282f-81b6-4518-b673-68f527fdfd04-tmp-dir\") pod \"distinct-w4-26743a-kserve-85985748cd-vl2fj\" (UID: \"93c5282f-81b6-4518-b673-68f527fdfd04\") " pod="e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj"
Oct 30 22:00:02.577640 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:02.577398    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjx47\" (UniqueName: \"kubernetes.io/projected/93c5282f-81b6-4518-b673-68f527fdfd04-kube-api-access-sjx47\") pod \"distinct-w4-26743a-kserve-85985748cd-vl2fj\" (UID: \"93c5282f-81b6-4518-b673-68f527fdfd04\") " pod="e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj"
Oct 30 22:00:02.601612 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:02.601581    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj"]
Oct 30 22:00:02.640798 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:02.640724    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-26743a\"/\"openshift-service-ca.crt\""
Oct 30 22:00:02.640798 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:02.640724    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-26743a\"/\"kube-root-ca.crt\""
Oct 30 22:00:02.641083 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:02.640854    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-26743a\"/\"default-dockercfg-c758n\""
Oct 30 22:00:02.645342 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:02.645319    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-26743a\"/\"distinct-w4-26743a-kserve-self-signed-certs\""
Oct 30 22:00:02.658888 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:02.658844    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:02.678990 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:02.678953    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93c5282f-81b6-4518-b673-68f527fdfd04-model-cache\") pod \"distinct-w4-26743a-kserve-85985748cd-vl2fj\" (UID: \"93c5282f-81b6-4518-b673-68f527fdfd04\") " pod="e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj"
Oct 30 22:00:02.680340 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:02.679586    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93c5282f-81b6-4518-b673-68f527fdfd04-dshm\") pod \"distinct-w4-26743a-kserve-85985748cd-vl2fj\" (UID: \"93c5282f-81b6-4518-b673-68f527fdfd04\") " pod="e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj"
Oct 30 22:00:02.681632 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:02.680784    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93c5282f-81b6-4518-b673-68f527fdfd04-tmp-dir\") pod \"distinct-w4-26743a-kserve-85985748cd-vl2fj\" (UID: \"93c5282f-81b6-4518-b673-68f527fdfd04\") " pod="e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj"
Oct 30 22:00:02.681857 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:02.681608    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93c5282f-81b6-4518-b673-68f527fdfd04-tmp-dir\") pod \"distinct-w4-26743a-kserve-85985748cd-vl2fj\" (UID: \"93c5282f-81b6-4518-b673-68f527fdfd04\") " pod="e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj"
Oct 30 22:00:02.682363 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:02.682343    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93c5282f-81b6-4518-b673-68f527fdfd04-model-cache\") pod \"distinct-w4-26743a-kserve-85985748cd-vl2fj\" (UID: \"93c5282f-81b6-4518-b673-68f527fdfd04\") " pod="e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj"
Oct 30 22:00:02.682689 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:02.682667    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjx47\" (UniqueName: \"kubernetes.io/projected/93c5282f-81b6-4518-b673-68f527fdfd04-kube-api-access-sjx47\") pod \"distinct-w4-26743a-kserve-85985748cd-vl2fj\" (UID: \"93c5282f-81b6-4518-b673-68f527fdfd04\") " pod="e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj"
Oct 30 22:00:02.683564 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:02.683545    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93c5282f-81b6-4518-b673-68f527fdfd04-home\") pod \"distinct-w4-26743a-kserve-85985748cd-vl2fj\" (UID: \"93c5282f-81b6-4518-b673-68f527fdfd04\") " pod="e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj"
Oct 30 22:00:02.683792 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:02.683774    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93c5282f-81b6-4518-b673-68f527fdfd04-home\") pod \"distinct-w4-26743a-kserve-85985748cd-vl2fj\" (UID: \"93c5282f-81b6-4518-b673-68f527fdfd04\") " pod="e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj"
Oct 30 22:00:02.684610 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:02.684591    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93c5282f-81b6-4518-b673-68f527fdfd04-tls-certs\") pod \"distinct-w4-26743a-kserve-85985748cd-vl2fj\" (UID: \"93c5282f-81b6-4518-b673-68f527fdfd04\") " pod="e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj"
Oct 30 22:00:02.699223 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:02.699103    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93c5282f-81b6-4518-b673-68f527fdfd04-tls-certs\") pod \"distinct-w4-26743a-kserve-85985748cd-vl2fj\" (UID: \"93c5282f-81b6-4518-b673-68f527fdfd04\") " pod="e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj"
Oct 30 22:00:02.714513 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:02.714440    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93c5282f-81b6-4518-b673-68f527fdfd04-dshm\") pod \"distinct-w4-26743a-kserve-85985748cd-vl2fj\" (UID: \"93c5282f-81b6-4518-b673-68f527fdfd04\") " pod="e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj"
Oct 30 22:00:02.761594 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:02.761494    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjx47\" (UniqueName: \"kubernetes.io/projected/93c5282f-81b6-4518-b673-68f527fdfd04-kube-api-access-sjx47\") pod \"distinct-w4-26743a-kserve-85985748cd-vl2fj\" (UID: \"93c5282f-81b6-4518-b673-68f527fdfd04\") " pod="e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj"
Oct 30 22:00:02.861507 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:02.861406    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj"
Oct 30 22:00:03.000623 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.000458    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:03.333643 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.333595    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9a5623-29889960-t8mjd" event={"ID":"4b6736b8-3237-4e98-8b56-132f67660bd1","Type":"ContainerStarted","Data":"5ec0ab46170a50c821fa9fd11efde98585c55bcbd975e263c6f29c9f21b8b88b"}
Oct 30 22:00:03.334139 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.333705    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9a5623-29889960-t8mjd" event={"ID":"4b6736b8-3237-4e98-8b56-132f67660bd1","Type":"ContainerStarted","Data":"0196fc00412aedfe9199aeaaec9bd76f8bdb19d2c746aecf3bf139677f972299"}
Oct 30 22:00:03.335975 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.335944    4871 generic.go:356] "Generic (PLEG): container finished" podID="ed194f68-2c40-49ae-b9a4-0680e645ecec" containerID="b4d0d1a70ef4830fd808b22d42e1b2546d90532a952bc21d2641ffe04003f5ed" exitCode=60
Oct 30 22:00:03.336105 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.336028    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889960-ng4ws" event={"ID":"ed194f68-2c40-49ae-b9a4-0680e645ecec","Type":"ContainerDied","Data":"b4d0d1a70ef4830fd808b22d42e1b2546d90532a952bc21d2641ffe04003f5ed"}
Oct 30 22:00:03.336322 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.336304    4871 scope.go:122] "RemoveContainer" containerID="b4d0d1a70ef4830fd808b22d42e1b2546d90532a952bc21d2641ffe04003f5ed"
Oct 30 22:00:03.348127 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.348096    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj"]
Oct 30 22:00:03.351639 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:00:03.351598    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod93c5282f_81b6_4518_b673_68f527fdfd04.slice/crio-2e1d917d7df390ba2af095cdd3fa122aa8b7a492cc203940ce9b63cd3cd4c8cf WatchSource:0}: Error finding container 2e1d917d7df390ba2af095cdd3fa122aa8b7a492cc203940ce9b63cd3cd4c8cf: Status 404 returned error can't find the container with id 2e1d917d7df390ba2af095cdd3fa122aa8b7a492cc203940ce9b63cd3cd4c8cf
Oct 30 22:00:03.477336 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.474886    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9a5623-29889960-t8mjd" podStartSLOduration=3.474868251 podStartE2EDuration="3.474868251s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:00:03.458781447 +0000 UTC m=+1345.636270087" watchObservedRunningTime="2026-10-30 22:00:03.474868251 +0000 UTC m=+1345.652356866"
Oct 30 22:00:03.559150 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.559104    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"]
Oct 30 22:00:03.564515 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.564474    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"
Oct 30 22:00:03.613261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.610136    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-istio-envoy\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-j6sfz\" (UID: \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"
Oct 30 22:00:03.613261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.610192    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-istio-podinfo\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-j6sfz\" (UID: \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"
Oct 30 22:00:03.613261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.610229    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9tbr\" (UniqueName: \"kubernetes.io/projected/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-kube-api-access-n9tbr\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-j6sfz\" (UID: \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"
Oct 30 22:00:03.613261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.610383    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-istiod-ca-cert\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-j6sfz\" (UID: \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"
Oct 30 22:00:03.613261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.610427    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-workload-socket\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-j6sfz\" (UID: \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"
Oct 30 22:00:03.613261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.610478    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-workload-certs\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-j6sfz\" (UID: \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"
Oct 30 22:00:03.613261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.610507    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-istio-data\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-j6sfz\" (UID: \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"
Oct 30 22:00:03.613261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.610600    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-credential-socket\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-j6sfz\" (UID: \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"
Oct 30 22:00:03.613261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.610624    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-istio-token\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-j6sfz\" (UID: \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"
Oct 30 22:00:03.643328 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.643291    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-4c260505-openshift-default-dockercfg-k4h5b\""
Oct 30 22:00:03.650252 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.650185    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"]
Oct 30 22:00:03.751388 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.725841    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-istio-envoy\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-j6sfz\" (UID: \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"
Oct 30 22:00:03.751668 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.751637    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-istio-podinfo\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-j6sfz\" (UID: \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"
Oct 30 22:00:03.751779 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.751766    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9tbr\" (UniqueName: \"kubernetes.io/projected/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-kube-api-access-n9tbr\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-j6sfz\" (UID: \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"
Oct 30 22:00:03.756374 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.751962    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-istiod-ca-cert\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-j6sfz\" (UID: \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"
Oct 30 22:00:03.765529 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.765377    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-workload-socket\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-j6sfz\" (UID: \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"
Oct 30 22:00:03.765966 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.765938    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-workload-certs\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-j6sfz\" (UID: \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"
Oct 30 22:00:03.778895 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.778855    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-istio-data\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-j6sfz\" (UID: \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"
Oct 30 22:00:03.808977 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.789264    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-istio-data\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-j6sfz\" (UID: \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"
Oct 30 22:00:03.808977 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.776635    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-workload-certs\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-j6sfz\" (UID: \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"
Oct 30 22:00:03.808977 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.768646    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-workload-socket\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-j6sfz\" (UID: \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"
Oct 30 22:00:03.808977 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.746176    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-istio-envoy\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-j6sfz\" (UID: \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"
Oct 30 22:00:03.808977 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.799880    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-istiod-ca-cert\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-j6sfz\" (UID: \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"
Oct 30 22:00:03.808977 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.779265    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-credential-socket\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-j6sfz\" (UID: \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"
Oct 30 22:00:03.808977 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.804996    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-istio-token\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-j6sfz\" (UID: \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"
Oct 30 22:00:03.808977 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.805892    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-credential-socket\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-j6sfz\" (UID: \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"
Oct 30 22:00:03.808977 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.808885    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:03.909663 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.909630    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-istio-token\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-j6sfz\" (UID: \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"
Oct 30 22:00:03.913643 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.913559    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-istio-podinfo\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-j6sfz\" (UID: \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"
Oct 30 22:00:03.934404 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.931481    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9tbr\" (UniqueName: \"kubernetes.io/projected/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-kube-api-access-n9tbr\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-j6sfz\" (UID: \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"
Oct 30 22:00:04.001815 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:03.992358    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:04.182286 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:04.182230    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"
Oct 30 22:00:04.347962 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:04.347925    4871 generic.go:356] "Generic (PLEG): container finished" podID="4b6736b8-3237-4e98-8b56-132f67660bd1" containerID="5ec0ab46170a50c821fa9fd11efde98585c55bcbd975e263c6f29c9f21b8b88b" exitCode=60
Oct 30 22:00:04.348483 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:04.348010    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9a5623-29889960-t8mjd" event={"ID":"4b6736b8-3237-4e98-8b56-132f67660bd1","Type":"ContainerDied","Data":"5ec0ab46170a50c821fa9fd11efde98585c55bcbd975e263c6f29c9f21b8b88b"}
Oct 30 22:00:04.348483 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:04.348312    4871 scope.go:122] "RemoveContainer" containerID="5ec0ab46170a50c821fa9fd11efde98585c55bcbd975e263c6f29c9f21b8b88b"
Oct 30 22:00:04.350319 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:04.350289    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889960-ng4ws" event={"ID":"ed194f68-2c40-49ae-b9a4-0680e645ecec","Type":"ContainerStarted","Data":"ec21948c2606293e84724bb1bed5198a4c7520c6e938d94cd54410b240367df0"}
Oct 30 22:00:04.352180 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:04.352158    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj" event={"ID":"93c5282f-81b6-4518-b673-68f527fdfd04","Type":"ContainerStarted","Data":"bd7d8b23763e165b81b338e03491a3abe248fc3bcd83d7c4b18849ff02d4396d"}
Oct 30 22:00:04.352303 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:04.352190    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj" event={"ID":"93c5282f-81b6-4518-b673-68f527fdfd04","Type":"ContainerStarted","Data":"2e1d917d7df390ba2af095cdd3fa122aa8b7a492cc203940ce9b63cd3cd4c8cf"}
Oct 30 22:00:04.575134 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:04.575075    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889960-ng4ws" podStartSLOduration=4.575059847 podStartE2EDuration="4.575059847s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:00:04.574409638 +0000 UTC m=+1346.751898254" watchObservedRunningTime="2026-10-30 22:00:04.575059847 +0000 UTC m=+1346.752548462"
Oct 30 22:00:04.620349 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:00:04.620110    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod198884dd_8c10_4d1f_9f64_6e00c0e7b6ab.slice/crio-ee53d1b1dc83a8ec70ec8053b84c8575a79789082c181c9829659b6dab06bb8d WatchSource:0}: Error finding container ee53d1b1dc83a8ec70ec8053b84c8575a79789082c181c9829659b6dab06bb8d: Status 404 returned error can't find the container with id ee53d1b1dc83a8ec70ec8053b84c8575a79789082c181c9829659b6dab06bb8d
Oct 30 22:00:04.620349 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:04.620126    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"]
Oct 30 22:00:04.622666 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:04.622635    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:00:04.622766 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:04.622705    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:00:04.622766 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:04.622734    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:00:04.685826 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:04.685775    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj" podStartSLOduration=2.6857603660000002 podStartE2EDuration="2.685760366s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:00:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:00:04.684436516 +0000 UTC m=+1346.861925135" watchObservedRunningTime="2026-10-30 22:00:04.685760366 +0000 UTC m=+1346.863248982"
Oct 30 22:00:04.771993 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:00:04.771904    4871 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33d005ac_b847_4fd5_af8f_5b0162b56610.slice\": RecentStats: unable to find data in memory cache]"
Oct 30 22:00:04.866263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:04.865259    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj"
Oct 30 22:00:05.363477 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:05.363438    4871 generic.go:356] "Generic (PLEG): container finished" podID="4b6736b8-3237-4e98-8b56-132f67660bd1" containerID="c31cff9fcb8140efbd5eb08d36e5feb3cc29808082f1cb14461a1a3c6c4221a6" exitCode=60
Oct 30 22:00:05.363956 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:05.363518    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9a5623-29889960-t8mjd" event={"ID":"4b6736b8-3237-4e98-8b56-132f67660bd1","Type":"ContainerDied","Data":"c31cff9fcb8140efbd5eb08d36e5feb3cc29808082f1cb14461a1a3c6c4221a6"}
Oct 30 22:00:05.363956 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:05.363593    4871 scope.go:122] "RemoveContainer" containerID="5ec0ab46170a50c821fa9fd11efde98585c55bcbd975e263c6f29c9f21b8b88b"
Oct 30 22:00:05.363956 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:05.363836    4871 scope.go:122] "RemoveContainer" containerID="c31cff9fcb8140efbd5eb08d36e5feb3cc29808082f1cb14461a1a3c6c4221a6"
Oct 30 22:00:05.364189 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:00:05.364165    4871 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-9a5623-29889960-t8mjd_odh-ai-gateway-infra(4b6736b8-3237-4e98-8b56-132f67660bd1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9a5623-29889960-t8mjd" podUID="4b6736b8-3237-4e98-8b56-132f67660bd1"
Oct 30 22:00:05.366038 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:05.366014    4871 generic.go:356] "Generic (PLEG): container finished" podID="ed194f68-2c40-49ae-b9a4-0680e645ecec" containerID="ec21948c2606293e84724bb1bed5198a4c7520c6e938d94cd54410b240367df0" exitCode=60
Oct 30 22:00:05.366135 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:05.366091    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889960-ng4ws" event={"ID":"ed194f68-2c40-49ae-b9a4-0680e645ecec","Type":"ContainerDied","Data":"ec21948c2606293e84724bb1bed5198a4c7520c6e938d94cd54410b240367df0"}
Oct 30 22:00:05.366392 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:05.366373    4871 scope.go:122] "RemoveContainer" containerID="ec21948c2606293e84724bb1bed5198a4c7520c6e938d94cd54410b240367df0"
Oct 30 22:00:05.366714 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:00:05.366687    4871 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-26743a-29889960-ng4ws_odh-ai-gateway-infra(ed194f68-2c40-49ae-b9a4-0680e645ecec)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889960-ng4ws" podUID="ed194f68-2c40-49ae-b9a4-0680e645ecec"
Oct 30 22:00:05.368178 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:05.368148    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz" event={"ID":"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab","Type":"ContainerStarted","Data":"b419997ba915969d7decd8f7a38ecdfe34dca49729e2b8747e3e95f3ea373138"}
Oct 30 22:00:05.368303 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:05.368213    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz" event={"ID":"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab","Type":"ContainerStarted","Data":"ee53d1b1dc83a8ec70ec8053b84c8575a79789082c181c9829659b6dab06bb8d"}
Oct 30 22:00:05.376786 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:05.376756    4871 scope.go:122] "RemoveContainer" containerID="b4d0d1a70ef4830fd808b22d42e1b2546d90532a952bc21d2641ffe04003f5ed"
Oct 30 22:00:05.569762 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:00:05.569666    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:00:05.597474 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:05.597416    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz" podStartSLOduration=2.5974010979999997 podStartE2EDuration="2.597401098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:00:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:00:05.593488965 +0000 UTC m=+1347.770977583" watchObservedRunningTime="2026-10-30 22:00:05.597401098 +0000 UTC m=+1347.774889713"
Oct 30 22:00:05.733730 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:05.733689    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:06.182626 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:06.182590    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"
Oct 30 22:00:06.189067 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:06.189034    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"
Oct 30 22:00:06.200371 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:00:06.200336    4871 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33d005ac_b847_4fd5_af8f_5b0162b56610.slice\": RecentStats: unable to find data in memory cache]"
Oct 30 22:00:06.378797 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:06.378764    4871 scope.go:122] "RemoveContainer" containerID="ec21948c2606293e84724bb1bed5198a4c7520c6e938d94cd54410b240367df0"
Oct 30 22:00:06.379233 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:00:06.379157    4871 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-26743a-29889960-ng4ws_odh-ai-gateway-infra(ed194f68-2c40-49ae-b9a4-0680e645ecec)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889960-ng4ws" podUID="ed194f68-2c40-49ae-b9a4-0680e645ecec"
Oct 30 22:00:06.380630 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:06.380602    4871 scope.go:122] "RemoveContainer" containerID="c31cff9fcb8140efbd5eb08d36e5feb3cc29808082f1cb14461a1a3c6c4221a6"
Oct 30 22:00:06.380757 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:06.380633    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"
Oct 30 22:00:06.380990 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:00:06.380950    4871 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-9a5623-29889960-t8mjd_odh-ai-gateway-infra(4b6736b8-3237-4e98-8b56-132f67660bd1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9a5623-29889960-t8mjd" podUID="4b6736b8-3237-4e98-8b56-132f67660bd1"
Oct 30 22:00:06.381600 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:06.381584    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"
Oct 30 22:00:09.090250 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:09.090201    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:09.293213 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:09.293171    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:10.668604 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:10.668566    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:12.861833 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:12.861792    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj"
Oct 30 22:00:12.874964 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:12.874935    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj"
Oct 30 22:00:13.443784 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:13.443749    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj"
Oct 30 22:00:15.611227 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:15.611193    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:15.953480 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:15.953445    4871 scope.go:122] "RemoveContainer" containerID="ec21948c2606293e84724bb1bed5198a4c7520c6e938d94cd54410b240367df0"
Oct 30 22:00:16.275835 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:00:16.275799    4871 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33d005ac_b847_4fd5_af8f_5b0162b56610.slice\": RecentStats: unable to find data in memory cache]"
Oct 30 22:00:16.458621 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:16.458586    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889960-ng4ws" event={"ID":"ed194f68-2c40-49ae-b9a4-0680e645ecec","Type":"ContainerStarted","Data":"371e48f146d8cebc44d50b66a3296c96a16cc0b4d923ac563e43d4570de3650d"}
Oct 30 22:00:17.346964 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:00:17.346924    4871 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33d005ac_b847_4fd5_af8f_5b0162b56610.slice\": RecentStats: unable to find data in memory cache]"
Oct 30 22:00:17.347416 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:00:17.347152    4871 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33d005ac_b847_4fd5_af8f_5b0162b56610.slice\": RecentStats: unable to find data in memory cache]"
Oct 30 22:00:17.470819 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:17.470781    4871 generic.go:356] "Generic (PLEG): container finished" podID="ed194f68-2c40-49ae-b9a4-0680e645ecec" containerID="371e48f146d8cebc44d50b66a3296c96a16cc0b4d923ac563e43d4570de3650d" exitCode=60
Oct 30 22:00:17.471154 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:17.471011    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889960-ng4ws" event={"ID":"ed194f68-2c40-49ae-b9a4-0680e645ecec","Type":"ContainerDied","Data":"371e48f146d8cebc44d50b66a3296c96a16cc0b4d923ac563e43d4570de3650d"}
Oct 30 22:00:17.471346 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:17.471209    4871 scope.go:122] "RemoveContainer" containerID="ec21948c2606293e84724bb1bed5198a4c7520c6e938d94cd54410b240367df0"
Oct 30 22:00:17.471718 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:17.471697    4871 scope.go:122] "RemoveContainer" containerID="371e48f146d8cebc44d50b66a3296c96a16cc0b4d923ac563e43d4570de3650d"
Oct 30 22:00:17.472122 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:00:17.472097    4871 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-26743a-29889960-ng4ws_odh-ai-gateway-infra(ed194f68-2c40-49ae-b9a4-0680e645ecec)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889960-ng4ws" podUID="ed194f68-2c40-49ae-b9a4-0680e645ecec"
Oct 30 22:00:17.593137 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:17.593096    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889960-ng4ws"]
Oct 30 22:00:17.958082 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:17.958052    4871 scope.go:122] "RemoveContainer" containerID="c31cff9fcb8140efbd5eb08d36e5feb3cc29808082f1cb14461a1a3c6c4221a6"
Oct 30 22:00:18.482096 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:18.481960    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9a5623-29889960-t8mjd" event={"ID":"4b6736b8-3237-4e98-8b56-132f67660bd1","Type":"ContainerStarted","Data":"2defb12b3b68ad504a253ecd98c802d96e6cd0c96e8cfb4687d805756bbd8d7d"}
Oct 30 22:00:18.544491 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:18.544223    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:18.666431 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:18.666391    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889960-ng4ws"
Oct 30 22:00:18.738460 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:18.738384    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ed194f68-2c40-49ae-b9a4-0680e645ecec-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed194f68-2c40-49ae-b9a4-0680e645ecec-maas-api-service-ca\") pod \"ed194f68-2c40-49ae-b9a4-0680e645ecec\" (UID: \"ed194f68-2c40-49ae-b9a4-0680e645ecec\") "
Oct 30 22:00:18.738460 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:18.738440    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ed194f68-2c40-49ae-b9a4-0680e645ecec-kube-api-access-6p4q5\" (UniqueName: \"kubernetes.io/projected/ed194f68-2c40-49ae-b9a4-0680e645ecec-kube-api-access-6p4q5\") pod \"ed194f68-2c40-49ae-b9a4-0680e645ecec\" (UID: \"ed194f68-2c40-49ae-b9a4-0680e645ecec\") "
Oct 30 22:00:18.738734 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:18.738705    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:18.738827 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:18.738802    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed194f68-2c40-49ae-b9a4-0680e645ecec-maas-api-service-ca" pod "ed194f68-2c40-49ae-b9a4-0680e645ecec" (UID: "ed194f68-2c40-49ae-b9a4-0680e645ecec"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:00:18.739135 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:18.739109    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed194f68-2c40-49ae-b9a4-0680e645ecec-maas-api-service-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:18.740909 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:18.740880    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed194f68-2c40-49ae-b9a4-0680e645ecec-kube-api-access-6p4q5" pod "ed194f68-2c40-49ae-b9a4-0680e645ecec" (UID: "ed194f68-2c40-49ae-b9a4-0680e645ecec"). InnerVolumeSpecName "kube-api-access-6p4q5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:00:18.840496 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:18.840462    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6p4q5\" (UniqueName: \"kubernetes.io/projected/ed194f68-2c40-49ae-b9a4-0680e645ecec-kube-api-access-6p4q5\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:19.018575 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:19.018485    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:19.493015 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:19.492983    4871 generic.go:356] "Generic (PLEG): container finished" podID="4b6736b8-3237-4e98-8b56-132f67660bd1" containerID="2defb12b3b68ad504a253ecd98c802d96e6cd0c96e8cfb4687d805756bbd8d7d" exitCode=60
Oct 30 22:00:19.493533 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:19.493053    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9a5623-29889960-t8mjd" event={"ID":"4b6736b8-3237-4e98-8b56-132f67660bd1","Type":"ContainerDied","Data":"2defb12b3b68ad504a253ecd98c802d96e6cd0c96e8cfb4687d805756bbd8d7d"}
Oct 30 22:00:19.493533 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:19.493127    4871 scope.go:122] "RemoveContainer" containerID="c31cff9fcb8140efbd5eb08d36e5feb3cc29808082f1cb14461a1a3c6c4221a6"
Oct 30 22:00:19.493684 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:19.493590    4871 scope.go:122] "RemoveContainer" containerID="2defb12b3b68ad504a253ecd98c802d96e6cd0c96e8cfb4687d805756bbd8d7d"
Oct 30 22:00:19.493939 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:00:19.493917    4871 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-9a5623-29889960-t8mjd_odh-ai-gateway-infra(4b6736b8-3237-4e98-8b56-132f67660bd1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9a5623-29889960-t8mjd" podUID="4b6736b8-3237-4e98-8b56-132f67660bd1"
Oct 30 22:00:19.494646 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:19.494627    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889960-ng4ws"
Oct 30 22:00:19.494719 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:19.494639    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889960-ng4ws" event={"ID":"ed194f68-2c40-49ae-b9a4-0680e645ecec","Type":"ContainerDied","Data":"1a861436245a062a45616327f9fdaa93e2f7fc3483c98252750df1f835dec767"}
Oct 30 22:00:19.510648 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:19.510622    4871 scope.go:122] "RemoveContainer" containerID="371e48f146d8cebc44d50b66a3296c96a16cc0b4d923ac563e43d4570de3650d"
Oct 30 22:00:19.611758 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:19.611723    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9a5623-29889960-t8mjd"]
Oct 30 22:00:19.636848 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:19.636815    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:19.695926 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:00:19.695893    4871 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33d005ac_b847_4fd5_af8f_5b0162b56610.slice\": RecentStats: unable to find data in memory cache]"
Oct 30 22:00:19.789224 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:19.789136    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889960-ng4ws"]
Oct 30 22:00:19.803372 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:19.803337    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889960-ng4ws"]
Oct 30 22:00:19.947227 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:19.947192    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:19.960585 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:19.960548    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ed194f68-2c40-49ae-b9a4-0680e645ecec" path="/var/lib/kubelet/pods/ed194f68-2c40-49ae-b9a4-0680e645ecec/volumes"
Oct 30 22:00:20.665871 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:20.665843    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9a5623-29889960-t8mjd"
Oct 30 22:00:20.763478 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:20.763377    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4b6736b8-3237-4e98-8b56-132f67660bd1-kube-api-access-lcsg5\" (UniqueName: \"kubernetes.io/projected/4b6736b8-3237-4e98-8b56-132f67660bd1-kube-api-access-lcsg5\") pod \"4b6736b8-3237-4e98-8b56-132f67660bd1\" (UID: \"4b6736b8-3237-4e98-8b56-132f67660bd1\") "
Oct 30 22:00:20.763665 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:20.763525    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4b6736b8-3237-4e98-8b56-132f67660bd1-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b6736b8-3237-4e98-8b56-132f67660bd1-maas-api-service-ca\") pod \"4b6736b8-3237-4e98-8b56-132f67660bd1\" (UID: \"4b6736b8-3237-4e98-8b56-132f67660bd1\") "
Oct 30 22:00:20.763935 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:20.763904    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b6736b8-3237-4e98-8b56-132f67660bd1-maas-api-service-ca" pod "4b6736b8-3237-4e98-8b56-132f67660bd1" (UID: "4b6736b8-3237-4e98-8b56-132f67660bd1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:00:20.764135 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:20.764113    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b6736b8-3237-4e98-8b56-132f67660bd1-maas-api-service-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:20.765896 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:20.765870    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b6736b8-3237-4e98-8b56-132f67660bd1-kube-api-access-lcsg5" pod "4b6736b8-3237-4e98-8b56-132f67660bd1" (UID: "4b6736b8-3237-4e98-8b56-132f67660bd1"). InnerVolumeSpecName "kube-api-access-lcsg5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:00:20.866083 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:20.866044    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lcsg5\" (UniqueName: \"kubernetes.io/projected/4b6736b8-3237-4e98-8b56-132f67660bd1-kube-api-access-lcsg5\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:20.996260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:20.990160    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:21.514773 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:21.514725    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9a5623-29889960-t8mjd"
Oct 30 22:00:21.514773 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:21.514753    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9a5623-29889960-t8mjd" event={"ID":"4b6736b8-3237-4e98-8b56-132f67660bd1","Type":"ContainerDied","Data":"0196fc00412aedfe9199aeaaec9bd76f8bdb19d2c746aecf3bf139677f972299"}
Oct 30 22:00:21.515040 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:21.514822    4871 scope.go:122] "RemoveContainer" containerID="2defb12b3b68ad504a253ecd98c802d96e6cd0c96e8cfb4687d805756bbd8d7d"
Oct 30 22:00:21.676795 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:21.676756    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9a5623-29889960-t8mjd"]
Oct 30 22:00:21.683822 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:21.683790    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9a5623-29889960-t8mjd"]
Oct 30 22:00:21.848120 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:21.848029    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:22.005258 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:21.999421    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4b6736b8-3237-4e98-8b56-132f67660bd1" path="/var/lib/kubelet/pods/4b6736b8-3237-4e98-8b56-132f67660bd1/volumes"
Oct 30 22:00:22.053261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.053135    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:22.423262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.416959    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-5tn88"]
Oct 30 22:00:22.423262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.417926    4871 cpu_manager.go:414] "containerMap: removing container" podUID="4b6736b8-3237-4e98-8b56-132f67660bd1" containerName="cleanup"
Oct 30 22:00:22.423262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.417949    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4b6736b8-3237-4e98-8b56-132f67660bd1" containerName="cleanup"
Oct 30 22:00:22.423262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.417962    4871 cpu_manager.go:414] "containerMap: removing container" podUID="4b6736b8-3237-4e98-8b56-132f67660bd1" containerName="cleanup"
Oct 30 22:00:22.423262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.417971    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4b6736b8-3237-4e98-8b56-132f67660bd1" containerName="cleanup"
Oct 30 22:00:22.423262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.418001    4871 cpu_manager.go:414] "containerMap: removing container" podUID="ed194f68-2c40-49ae-b9a4-0680e645ecec" containerName="cleanup"
Oct 30 22:00:22.423262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.418010    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ed194f68-2c40-49ae-b9a4-0680e645ecec" containerName="cleanup"
Oct 30 22:00:22.423262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.418022    4871 cpu_manager.go:414] "containerMap: removing container" podUID="ed194f68-2c40-49ae-b9a4-0680e645ecec" containerName="cleanup"
Oct 30 22:00:22.423262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.418030    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ed194f68-2c40-49ae-b9a4-0680e645ecec" containerName="cleanup"
Oct 30 22:00:22.423262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.418209    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="ed194f68-2c40-49ae-b9a4-0680e645ecec" containerName="cleanup"
Oct 30 22:00:22.423262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.418224    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="ed194f68-2c40-49ae-b9a4-0680e645ecec" containerName="cleanup"
Oct 30 22:00:22.432674 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.430485    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="ed194f68-2c40-49ae-b9a4-0680e645ecec" containerName="cleanup"
Oct 30 22:00:22.432674 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.430531    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="4b6736b8-3237-4e98-8b56-132f67660bd1" containerName="cleanup"
Oct 30 22:00:22.432674 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.430557    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="4b6736b8-3237-4e98-8b56-132f67660bd1" containerName="cleanup"
Oct 30 22:00:22.432674 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.430565    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="4b6736b8-3237-4e98-8b56-132f67660bd1" containerName="cleanup"
Oct 30 22:00:22.449302 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.437477    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-5tn88"
Oct 30 22:00:22.481259 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.478440    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-5tn88"]
Oct 30 22:00:22.495827 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.491210    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-4c260505\""
Oct 30 22:00:22.520762 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.516265    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-4c260505-dockercfg-h2wfg\""
Oct 30 22:00:22.540168 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.540129    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mjhq\" (UniqueName: \"kubernetes.io/projected/4316833e-6973-4742-9eda-fb87f445a73b-kube-api-access-4mjhq\") pod \"payload-processing-e2e-mt-4c260505-68cf845c4f-5tn88\" (UID: \"4316833e-6973-4742-9eda-fb87f445a73b\") " pod="openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-5tn88"
Oct 30 22:00:22.540476 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.540453    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4316833e-6973-4742-9eda-fb87f445a73b-plugins-config-volume\") pod \"payload-processing-e2e-mt-4c260505-68cf845c4f-5tn88\" (UID: \"4316833e-6973-4742-9eda-fb87f445a73b\") " pod="openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-5tn88"
Oct 30 22:00:22.634650 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.634612    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4c260505-65bb6f8647-p5g2x"]
Oct 30 22:00:22.636036 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.636009    4871 cpu_manager.go:414] "containerMap: removing container" podUID="4b6736b8-3237-4e98-8b56-132f67660bd1" containerName="cleanup"
Oct 30 22:00:22.636036 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.636037    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4b6736b8-3237-4e98-8b56-132f67660bd1" containerName="cleanup"
Oct 30 22:00:22.636177 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.636098    4871 cpu_manager.go:414] "containerMap: removing container" podUID="ed194f68-2c40-49ae-b9a4-0680e645ecec" containerName="cleanup"
Oct 30 22:00:22.636177 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.636106    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ed194f68-2c40-49ae-b9a4-0680e645ecec" containerName="cleanup"
Oct 30 22:00:22.644259 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.644203    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mjhq\" (UniqueName: \"kubernetes.io/projected/4316833e-6973-4742-9eda-fb87f445a73b-kube-api-access-4mjhq\") pod \"payload-processing-e2e-mt-4c260505-68cf845c4f-5tn88\" (UID: \"4316833e-6973-4742-9eda-fb87f445a73b\") " pod="openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-5tn88"
Oct 30 22:00:22.644482 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.644321    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4316833e-6973-4742-9eda-fb87f445a73b-plugins-config-volume\") pod \"payload-processing-e2e-mt-4c260505-68cf845c4f-5tn88\" (UID: \"4316833e-6973-4742-9eda-fb87f445a73b\") " pod="openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-5tn88"
Oct 30 22:00:22.644866 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.644840    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4c260505-65bb6f8647-p5g2x"]
Oct 30 22:00:22.644991 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.644976    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-4c260505-65bb6f8647-p5g2x"
Oct 30 22:00:22.654294 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.654262    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4316833e-6973-4742-9eda-fb87f445a73b-plugins-config-volume\") pod \"payload-processing-e2e-mt-4c260505-68cf845c4f-5tn88\" (UID: \"4316833e-6973-4742-9eda-fb87f445a73b\") " pod="openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-5tn88"
Oct 30 22:00:22.713271 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.713178    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mjhq\" (UniqueName: \"kubernetes.io/projected/4316833e-6973-4742-9eda-fb87f445a73b-kube-api-access-4mjhq\") pod \"payload-processing-e2e-mt-4c260505-68cf845c4f-5tn88\" (UID: \"4316833e-6973-4742-9eda-fb87f445a73b\") " pod="openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-5tn88"
Oct 30 22:00:22.745729 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.745686    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mspfh\" (UniqueName: \"kubernetes.io/projected/d1717354-d285-4b73-a85e-8e8af064463b-kube-api-access-mspfh\") pod \"payload-pre-processing-e2e-mt-4c260505-65bb6f8647-p5g2x\" (UID: \"d1717354-d285-4b73-a85e-8e8af064463b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4c260505-65bb6f8647-p5g2x"
Oct 30 22:00:22.745936 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.745916    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1717354-d285-4b73-a85e-8e8af064463b-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-4c260505-65bb6f8647-p5g2x\" (UID: \"d1717354-d285-4b73-a85e-8e8af064463b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4c260505-65bb6f8647-p5g2x"
Oct 30 22:00:22.799892 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.799857    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-5tn88"
Oct 30 22:00:22.846953 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.846902    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1717354-d285-4b73-a85e-8e8af064463b-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-4c260505-65bb6f8647-p5g2x\" (UID: \"d1717354-d285-4b73-a85e-8e8af064463b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4c260505-65bb6f8647-p5g2x"
Oct 30 22:00:22.847160 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.847014    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mspfh\" (UniqueName: \"kubernetes.io/projected/d1717354-d285-4b73-a85e-8e8af064463b-kube-api-access-mspfh\") pod \"payload-pre-processing-e2e-mt-4c260505-65bb6f8647-p5g2x\" (UID: \"d1717354-d285-4b73-a85e-8e8af064463b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4c260505-65bb6f8647-p5g2x"
Oct 30 22:00:22.847985 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.847958    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1717354-d285-4b73-a85e-8e8af064463b-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-4c260505-65bb6f8647-p5g2x\" (UID: \"d1717354-d285-4b73-a85e-8e8af064463b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4c260505-65bb6f8647-p5g2x"
Oct 30 22:00:22.906506 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.906467    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mspfh\" (UniqueName: \"kubernetes.io/projected/d1717354-d285-4b73-a85e-8e8af064463b-kube-api-access-mspfh\") pod \"payload-pre-processing-e2e-mt-4c260505-65bb6f8647-p5g2x\" (UID: \"d1717354-d285-4b73-a85e-8e8af064463b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4c260505-65bb6f8647-p5g2x"
Oct 30 22:00:22.966298 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:22.966162    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-4c260505-65bb6f8647-p5g2x"
Oct 30 22:00:23.179947 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:23.179910    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-5tn88"]
Oct 30 22:00:23.181572 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:00:23.181523    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4316833e_6973_4742_9eda_fb87f445a73b.slice/crio-6ddb3579a370156e8fd9b6fdfe2fc8bff8426605ae4dd8c8c4060cf21f6088f3 WatchSource:0}: Error finding container 6ddb3579a370156e8fd9b6fdfe2fc8bff8426605ae4dd8c8c4060cf21f6088f3: Status 404 returned error can't find the container with id 6ddb3579a370156e8fd9b6fdfe2fc8bff8426605ae4dd8c8c4060cf21f6088f3
Oct 30 22:00:23.348165 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:23.348138    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4c260505-65bb6f8647-p5g2x"]
Oct 30 22:00:23.350304 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:00:23.350272    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1717354_d285_4b73_a85e_8e8af064463b.slice/crio-4649c124ae7c7d8a521b7e45e04adf02793a4e711a12c5d9b09fa025db67f79f WatchSource:0}: Error finding container 4649c124ae7c7d8a521b7e45e04adf02793a4e711a12c5d9b09fa025db67f79f: Status 404 returned error can't find the container with id 4649c124ae7c7d8a521b7e45e04adf02793a4e711a12c5d9b09fa025db67f79f
Oct 30 22:00:23.578095 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:23.577999    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-4c260505-65bb6f8647-p5g2x" event={"ID":"d1717354-d285-4b73-a85e-8e8af064463b","Type":"ContainerStarted","Data":"8021e9fd80d19ec1f0e6c34b6eb348cc6bd78c7ba8cfbbcad57297803d83fdb0"}
Oct 30 22:00:23.578095 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:23.578050    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-4c260505-65bb6f8647-p5g2x" event={"ID":"d1717354-d285-4b73-a85e-8e8af064463b","Type":"ContainerStarted","Data":"4649c124ae7c7d8a521b7e45e04adf02793a4e711a12c5d9b09fa025db67f79f"}
Oct 30 22:00:23.578353 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:23.578201    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-4c260505-65bb6f8647-p5g2x"
Oct 30 22:00:23.579865 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:23.579839    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-5tn88" event={"ID":"4316833e-6973-4742-9eda-fb87f445a73b","Type":"ContainerStarted","Data":"dfec834e8e05069bd20d95ea2565f3930e1af456049ed8c91ddf047e278ed86c"}
Oct 30 22:00:23.579988 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:23.579879    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-5tn88" event={"ID":"4316833e-6973-4742-9eda-fb87f445a73b","Type":"ContainerStarted","Data":"6ddb3579a370156e8fd9b6fdfe2fc8bff8426605ae4dd8c8c4060cf21f6088f3"}
Oct 30 22:00:23.579988 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:23.579937    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-5tn88"
Oct 30 22:00:23.690634 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:23.690576    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-4c260505-65bb6f8647-p5g2x" podStartSLOduration=1.690559163 podStartE2EDuration="1.690559163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:00:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:00:23.688719954 +0000 UTC m=+1365.866208574" watchObservedRunningTime="2026-10-30 22:00:23.690559163 +0000 UTC m=+1365.868047823"
Oct 30 22:00:23.801811 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:23.801759    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-5tn88" podStartSLOduration=1.801745032 podStartE2EDuration="1.801745032s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:00:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:00:23.800918601 +0000 UTC m=+1365.978407217" watchObservedRunningTime="2026-10-30 22:00:23.801745032 +0000 UTC m=+1365.979233648"
Oct 30 22:00:24.767589 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:24.767549    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"]
Oct 30 22:00:24.772300 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:24.772272    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"
Oct 30 22:00:24.816657 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:24.816614    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"]
Oct 30 22:00:24.822324 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:24.822280    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"
Oct 30 22:00:24.872078 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:24.872035    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"]
Oct 30 22:00:24.883784 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:24.883748    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f3ddcc7-e832-42e9-97d4-1541f5a09044-workload-certs\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd\" (UID: \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"
Oct 30 22:00:24.883986 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:24.883799    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f3ddcc7-e832-42e9-97d4-1541f5a09044-istio-envoy\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd\" (UID: \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"
Oct 30 22:00:24.883986 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:24.883950    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f3ddcc7-e832-42e9-97d4-1541f5a09044-credential-socket\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd\" (UID: \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"
Oct 30 22:00:24.884090 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:24.884026    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f3ddcc7-e832-42e9-97d4-1541f5a09044-istio-data\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd\" (UID: \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"
Oct 30 22:00:24.884135 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:24.884089    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f3ddcc7-e832-42e9-97d4-1541f5a09044-istio-podinfo\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd\" (UID: \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"
Oct 30 22:00:24.884298 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:24.884276    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nd4kq\" (UniqueName: \"kubernetes.io/projected/5f3ddcc7-e832-42e9-97d4-1541f5a09044-kube-api-access-nd4kq\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd\" (UID: \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"
Oct 30 22:00:24.884330 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:24.884317    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f3ddcc7-e832-42e9-97d4-1541f5a09044-istiod-ca-cert\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd\" (UID: \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"
Oct 30 22:00:24.884426 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:24.884411    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f3ddcc7-e832-42e9-97d4-1541f5a09044-istio-token\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd\" (UID: \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"
Oct 30 22:00:24.884540 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:24.884522    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f3ddcc7-e832-42e9-97d4-1541f5a09044-workload-socket\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd\" (UID: \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"
Oct 30 22:00:24.928883 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:24.927104    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"]
Oct 30 22:00:24.986447 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:24.986409    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f3ddcc7-e832-42e9-97d4-1541f5a09044-istio-token\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd\" (UID: \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"
Oct 30 22:00:24.986447 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:24.986453    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84b9ee78-064c-4893-b81f-b9013b323448-istio-data\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq\" (UID: \"84b9ee78-064c-4893-b81f-b9013b323448\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"
Oct 30 22:00:24.986707 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:24.986472    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84b9ee78-064c-4893-b81f-b9013b323448-istio-envoy\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq\" (UID: \"84b9ee78-064c-4893-b81f-b9013b323448\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"
Oct 30 22:00:24.986707 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:24.986634    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f3ddcc7-e832-42e9-97d4-1541f5a09044-workload-socket\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd\" (UID: \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"
Oct 30 22:00:24.986707 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:24.986680    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84b9ee78-064c-4893-b81f-b9013b323448-workload-socket\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq\" (UID: \"84b9ee78-064c-4893-b81f-b9013b323448\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"
Oct 30 22:00:24.986707 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:24.986701    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84b9ee78-064c-4893-b81f-b9013b323448-credential-socket\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq\" (UID: \"84b9ee78-064c-4893-b81f-b9013b323448\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"
Oct 30 22:00:24.986906 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:24.986717    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84b9ee78-064c-4893-b81f-b9013b323448-istiod-ca-cert\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq\" (UID: \"84b9ee78-064c-4893-b81f-b9013b323448\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"
Oct 30 22:00:24.986906 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:24.986757    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f3ddcc7-e832-42e9-97d4-1541f5a09044-workload-certs\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd\" (UID: \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"
Oct 30 22:00:24.986906 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:24.986889    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f3ddcc7-e832-42e9-97d4-1541f5a09044-istio-envoy\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd\" (UID: \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"
Oct 30 22:00:24.987088 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:24.986934    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84b9ee78-064c-4893-b81f-b9013b323448-workload-certs\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq\" (UID: \"84b9ee78-064c-4893-b81f-b9013b323448\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"
Oct 30 22:00:24.987088 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:24.986987    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f3ddcc7-e832-42e9-97d4-1541f5a09044-credential-socket\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd\" (UID: \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"
Oct 30 22:00:24.987088 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:24.987011    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f3ddcc7-e832-42e9-97d4-1541f5a09044-workload-socket\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd\" (UID: \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"
Oct 30 22:00:24.987088 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:24.987024    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f3ddcc7-e832-42e9-97d4-1541f5a09044-istio-data\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd\" (UID: \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"
Oct 30 22:00:24.987307 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:24.987084    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f3ddcc7-e832-42e9-97d4-1541f5a09044-istio-podinfo\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd\" (UID: \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"
Oct 30 22:00:24.987307 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:24.987110    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f3ddcc7-e832-42e9-97d4-1541f5a09044-workload-certs\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd\" (UID: \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"
Oct 30 22:00:24.987307 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:24.987136    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84b9ee78-064c-4893-b81f-b9013b323448-istio-token\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq\" (UID: \"84b9ee78-064c-4893-b81f-b9013b323448\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"
Oct 30 22:00:24.987307 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:24.987168    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84b9ee78-064c-4893-b81f-b9013b323448-istio-podinfo\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq\" (UID: \"84b9ee78-064c-4893-b81f-b9013b323448\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"
Oct 30 22:00:24.987498 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:24.987335    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f3ddcc7-e832-42e9-97d4-1541f5a09044-istio-data\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd\" (UID: \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"
Oct 30 22:00:24.987498 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:24.987352    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99h6f\" (UniqueName: \"kubernetes.io/projected/84b9ee78-064c-4893-b81f-b9013b323448-kube-api-access-99h6f\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq\" (UID: \"84b9ee78-064c-4893-b81f-b9013b323448\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"
Oct 30 22:00:24.987498 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:24.987449    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nd4kq\" (UniqueName: \"kubernetes.io/projected/5f3ddcc7-e832-42e9-97d4-1541f5a09044-kube-api-access-nd4kq\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd\" (UID: \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"
Oct 30 22:00:24.987498 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:24.987485    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f3ddcc7-e832-42e9-97d4-1541f5a09044-istiod-ca-cert\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd\" (UID: \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"
Oct 30 22:00:24.987694 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:24.987612    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f3ddcc7-e832-42e9-97d4-1541f5a09044-credential-socket\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd\" (UID: \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"
Oct 30 22:00:24.988328 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:24.988300    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f3ddcc7-e832-42e9-97d4-1541f5a09044-istiod-ca-cert\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd\" (UID: \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"
Oct 30 22:00:24.989273 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:24.989228    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f3ddcc7-e832-42e9-97d4-1541f5a09044-istio-envoy\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd\" (UID: \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"
Oct 30 22:00:24.989760 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:24.989742    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f3ddcc7-e832-42e9-97d4-1541f5a09044-istio-podinfo\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd\" (UID: \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"
Oct 30 22:00:25.051328 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:25.047292    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f3ddcc7-e832-42e9-97d4-1541f5a09044-istio-token\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd\" (UID: \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"
Oct 30 22:00:25.051328 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:25.048092    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nd4kq\" (UniqueName: \"kubernetes.io/projected/5f3ddcc7-e832-42e9-97d4-1541f5a09044-kube-api-access-nd4kq\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd\" (UID: \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"
Oct 30 22:00:25.087857 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:25.087819    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"
Oct 30 22:00:25.088415 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:25.088391    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84b9ee78-064c-4893-b81f-b9013b323448-workload-socket\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq\" (UID: \"84b9ee78-064c-4893-b81f-b9013b323448\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"
Oct 30 22:00:25.088488 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:25.088424    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84b9ee78-064c-4893-b81f-b9013b323448-credential-socket\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq\" (UID: \"84b9ee78-064c-4893-b81f-b9013b323448\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"
Oct 30 22:00:25.088488 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:25.088441    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84b9ee78-064c-4893-b81f-b9013b323448-istiod-ca-cert\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq\" (UID: \"84b9ee78-064c-4893-b81f-b9013b323448\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"
Oct 30 22:00:25.088488 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:25.088486    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84b9ee78-064c-4893-b81f-b9013b323448-workload-certs\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq\" (UID: \"84b9ee78-064c-4893-b81f-b9013b323448\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"
Oct 30 22:00:25.088606 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:25.088563    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84b9ee78-064c-4893-b81f-b9013b323448-istio-token\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq\" (UID: \"84b9ee78-064c-4893-b81f-b9013b323448\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"
Oct 30 22:00:25.088606 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:25.088587    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84b9ee78-064c-4893-b81f-b9013b323448-istio-podinfo\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq\" (UID: \"84b9ee78-064c-4893-b81f-b9013b323448\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"
Oct 30 22:00:25.088695 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:25.088645    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99h6f\" (UniqueName: \"kubernetes.io/projected/84b9ee78-064c-4893-b81f-b9013b323448-kube-api-access-99h6f\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq\" (UID: \"84b9ee78-064c-4893-b81f-b9013b323448\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"
Oct 30 22:00:25.088751 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:25.088737    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84b9ee78-064c-4893-b81f-b9013b323448-istio-data\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq\" (UID: \"84b9ee78-064c-4893-b81f-b9013b323448\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"
Oct 30 22:00:25.088803 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:25.088766    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84b9ee78-064c-4893-b81f-b9013b323448-istio-envoy\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq\" (UID: \"84b9ee78-064c-4893-b81f-b9013b323448\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"
Oct 30 22:00:25.088865 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:25.088838    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84b9ee78-064c-4893-b81f-b9013b323448-credential-socket\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq\" (UID: \"84b9ee78-064c-4893-b81f-b9013b323448\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"
Oct 30 22:00:25.089155 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:25.089125    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84b9ee78-064c-4893-b81f-b9013b323448-workload-socket\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq\" (UID: \"84b9ee78-064c-4893-b81f-b9013b323448\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"
Oct 30 22:00:25.089303 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:25.089274    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84b9ee78-064c-4893-b81f-b9013b323448-istiod-ca-cert\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq\" (UID: \"84b9ee78-064c-4893-b81f-b9013b323448\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"
Oct 30 22:00:25.089417 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:25.089394    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84b9ee78-064c-4893-b81f-b9013b323448-istio-data\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq\" (UID: \"84b9ee78-064c-4893-b81f-b9013b323448\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"
Oct 30 22:00:25.089474 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:25.089437    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84b9ee78-064c-4893-b81f-b9013b323448-workload-certs\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq\" (UID: \"84b9ee78-064c-4893-b81f-b9013b323448\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"
Oct 30 22:00:25.091284 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:25.091266    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84b9ee78-064c-4893-b81f-b9013b323448-istio-envoy\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq\" (UID: \"84b9ee78-064c-4893-b81f-b9013b323448\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"
Oct 30 22:00:25.091597 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:25.091576    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84b9ee78-064c-4893-b81f-b9013b323448-istio-podinfo\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq\" (UID: \"84b9ee78-064c-4893-b81f-b9013b323448\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"
Oct 30 22:00:25.152739 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:25.152707    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99h6f\" (UniqueName: \"kubernetes.io/projected/84b9ee78-064c-4893-b81f-b9013b323448-kube-api-access-99h6f\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq\" (UID: \"84b9ee78-064c-4893-b81f-b9013b323448\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"
Oct 30 22:00:25.153150 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:25.153123    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84b9ee78-064c-4893-b81f-b9013b323448-istio-token\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq\" (UID: \"84b9ee78-064c-4893-b81f-b9013b323448\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"
Oct 30 22:00:25.439498 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:25.439460    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"
Oct 30 22:00:25.535786 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:25.535749    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"]
Oct 30 22:00:25.540722 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:00:25.540685    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f3ddcc7_e832_42e9_97d4_1541f5a09044.slice/crio-7c0cb35df4e8fd10fd40d9d717ab67c7320af06814263396b5640586d2c31f37 WatchSource:0}: Error finding container 7c0cb35df4e8fd10fd40d9d717ab67c7320af06814263396b5640586d2c31f37: Status 404 returned error can't find the container with id 7c0cb35df4e8fd10fd40d9d717ab67c7320af06814263396b5640586d2c31f37
Oct 30 22:00:25.544219 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:25.544181    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:00:25.544325 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:25.544289    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:00:25.544373 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:25.544334    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:00:25.597188 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:25.597155    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd" event={"ID":"5f3ddcc7-e832-42e9-97d4-1541f5a09044","Type":"ContainerStarted","Data":"7c0cb35df4e8fd10fd40d9d717ab67c7320af06814263396b5640586d2c31f37"}
Oct 30 22:00:25.734553 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:25.734466    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:25.863652 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:25.862559    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:00:25.863652 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:25.862678    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:00:25.863652 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:25.862728    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:00:25.875519 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:25.874531    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"]
Oct 30 22:00:26.297385 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:00:26.297343    4871 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33d005ac_b847_4fd5_af8f_5b0162b56610.slice\": RecentStats: unable to find data in memory cache]"
Oct 30 22:00:26.635264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:26.629623    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq" event={"ID":"84b9ee78-064c-4893-b81f-b9013b323448","Type":"ContainerStarted","Data":"f5735ba1aee7bc07b2251bca02fbcb206117a930dffc6ff7788239231f62d947"}
Oct 30 22:00:26.635264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:26.629702    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq" event={"ID":"84b9ee78-064c-4893-b81f-b9013b323448","Type":"ContainerStarted","Data":"1d4712689e481871c4df97e3a3e51beb73f20711713f00e33d72ee8edc9305be"}
Oct 30 22:00:26.635264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:26.633410    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd" event={"ID":"5f3ddcc7-e832-42e9-97d4-1541f5a09044","Type":"ContainerStarted","Data":"b7cc2da8b56f98b3003f2c936c4c963d28a532ad010e7069ae24ec67b74aa620"}
Oct 30 22:00:26.795440 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:26.795374    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq" podStartSLOduration=2.795355808 podStartE2EDuration="2.795355808s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:00:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:00:26.79463024 +0000 UTC m=+1368.972118858" watchObservedRunningTime="2026-10-30 22:00:26.795355808 +0000 UTC m=+1368.972844425"
Oct 30 22:00:26.939054 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:26.938928    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd" podStartSLOduration=2.938911141 podStartE2EDuration="2.938911141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:00:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:00:26.931859176 +0000 UTC m=+1369.109347818" watchObservedRunningTime="2026-10-30 22:00:26.938911141 +0000 UTC m=+1369.116399757"
Oct 30 22:00:27.088204 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:27.088160    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"
Oct 30 22:00:27.090048 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:27.090022    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.108:15021/healthz/ready\": dial tcp 10.133.0.108:15021: connect: connection refused" start-of-body=
Oct 30 22:00:27.090167 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:27.090072    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd" podUID="5f3ddcc7-e832-42e9-97d4-1541f5a09044" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.108:15021/healthz/ready\": dial tcp 10.133.0.108:15021: connect: connection refused"
Oct 30 22:00:27.353924 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:27.353827    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:27.447300 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:27.442812    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"
Oct 30 22:00:27.447300 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:27.442991    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.109:15021/healthz/ready\": dial tcp 10.133.0.109:15021: connect: connection refused" start-of-body=
Oct 30 22:00:27.447300 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:27.443041    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq" podUID="84b9ee78-064c-4893-b81f-b9013b323448" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.109:15021/healthz/ready\": dial tcp 10.133.0.109:15021: connect: connection refused"
Oct 30 22:00:28.088963 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:28.088916    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.108:15021/healthz/ready\": dial tcp 10.133.0.108:15021: connect: connection refused" start-of-body=
Oct 30 22:00:28.089393 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:28.088981    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd" podUID="5f3ddcc7-e832-42e9-97d4-1541f5a09044" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.108:15021/healthz/ready\": dial tcp 10.133.0.108:15021: connect: connection refused"
Oct 30 22:00:28.449263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:28.443687    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.109:15021/healthz/ready\": dial tcp 10.133.0.109:15021: connect: connection refused" start-of-body=
Oct 30 22:00:28.449263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:28.443760    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq" podUID="84b9ee78-064c-4893-b81f-b9013b323448" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.109:15021/healthz/ready\": dial tcp 10.133.0.109:15021: connect: connection refused"
Oct 30 22:00:29.089228 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:29.089192    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.108:15021/healthz/ready\": dial tcp 10.133.0.108:15021: connect: connection refused" start-of-body=
Oct 30 22:00:29.089670 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:29.089260    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd" podUID="5f3ddcc7-e832-42e9-97d4-1541f5a09044" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.108:15021/healthz/ready\": dial tcp 10.133.0.108:15021: connect: connection refused"
Oct 30 22:00:29.446262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:29.441674    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.109:15021/healthz/ready\": dial tcp 10.133.0.109:15021: connect: connection refused" start-of-body=
Oct 30 22:00:29.446262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:29.441737    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq" podUID="84b9ee78-064c-4893-b81f-b9013b323448" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.109:15021/healthz/ready\": dial tcp 10.133.0.109:15021: connect: connection refused"
Oct 30 22:00:30.100264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:30.090657    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.108:15021/healthz/ready\": dial tcp 10.133.0.108:15021: connect: connection refused" start-of-body=
Oct 30 22:00:30.100264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:30.090718    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd" podUID="5f3ddcc7-e832-42e9-97d4-1541f5a09044" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.108:15021/healthz/ready\": dial tcp 10.133.0.108:15021: connect: connection refused"
Oct 30 22:00:30.454261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:30.443613    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.109:15021/healthz/ready\": dial tcp 10.133.0.109:15021: connect: connection refused" start-of-body=
Oct 30 22:00:30.454261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:30.443677    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq" podUID="84b9ee78-064c-4893-b81f-b9013b323448" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.109:15021/healthz/ready\": dial tcp 10.133.0.109:15021: connect: connection refused"
Oct 30 22:00:30.840071 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:30.839940    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:31.101819 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:31.101717    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.108:15021/healthz/ready\": dial tcp 10.133.0.108:15021: connect: connection refused" start-of-body=
Oct 30 22:00:31.103381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:31.103344    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd" podUID="5f3ddcc7-e832-42e9-97d4-1541f5a09044" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.108:15021/healthz/ready\": dial tcp 10.133.0.108:15021: connect: connection refused"
Oct 30 22:00:31.255482 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:31.255402    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8" podUID="84329f90-dc35-41dc-b593-12c556e80f73" containerName="main" probeResult="failure" output="Get \"https://10.133.0.83:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:00:31.444004 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:31.443963    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.109:15021/healthz/ready\": dial tcp 10.133.0.109:15021: connect: connection refused" start-of-body=
Oct 30 22:00:31.444791 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:31.444758    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq" podUID="84b9ee78-064c-4893-b81f-b9013b323448" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.109:15021/healthz/ready\": dial tcp 10.133.0.109:15021: connect: connection refused"
Oct 30 22:00:31.533788 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:31.533748    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:32.091700 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:32.091661    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.108:15021/healthz/ready\": dial tcp 10.133.0.108:15021: connect: connection refused" start-of-body=
Oct 30 22:00:32.091982 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:32.091949    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd" podUID="5f3ddcc7-e832-42e9-97d4-1541f5a09044" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.108:15021/healthz/ready\": dial tcp 10.133.0.108:15021: connect: connection refused"
Oct 30 22:00:32.444264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:32.440669    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.109:15021/healthz/ready\": dial tcp 10.133.0.109:15021: connect: connection refused" start-of-body=
Oct 30 22:00:32.444264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:32.440755    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq" podUID="84b9ee78-064c-4893-b81f-b9013b323448" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.109:15021/healthz/ready\": dial tcp 10.133.0.109:15021: connect: connection refused"
Oct 30 22:00:32.448272 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:32.445325    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj" podUID="93c5282f-81b6-4518-b673-68f527fdfd04" containerName="main" probeResult="failure" output="Get \"https://10.133.0.104:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:00:33.089726 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:33.089689    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.108:15021/healthz/ready\": dial tcp 10.133.0.108:15021: connect: connection refused" start-of-body=
Oct 30 22:00:33.089997 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:33.089970    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd" podUID="5f3ddcc7-e832-42e9-97d4-1541f5a09044" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.108:15021/healthz/ready\": dial tcp 10.133.0.108:15021: connect: connection refused"
Oct 30 22:00:33.290435 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:33.290377    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8" podUID="84329f90-dc35-41dc-b593-12c556e80f73" containerName="main" probeResult="failure" output="Get \"https://10.133.0.83:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:00:33.443692 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:33.443646    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.109:15021/healthz/ready\": dial tcp 10.133.0.109:15021: connect: connection refused" start-of-body=
Oct 30 22:00:33.443883 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:33.443715    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq" podUID="84b9ee78-064c-4893-b81f-b9013b323448" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.109:15021/healthz/ready\": dial tcp 10.133.0.109:15021: connect: connection refused"
Oct 30 22:00:33.456399 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:33.454323    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj" podUID="93c5282f-81b6-4518-b673-68f527fdfd04" containerName="main" probeResult="failure" output="Get \"https://10.133.0.104:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:00:33.456399 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:33.454789    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj"
Oct 30 22:00:33.726372 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:33.718823    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:34.001469 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:34.001339    4871 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj" podUID="93c5282f-81b6-4518-b673-68f527fdfd04" containerName="main" probeResult="failure" output="Get \"https://10.133.0.104:8000/health\": context deadline exceeded"
Oct 30 22:00:34.128582 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:34.128545    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.108:15021/healthz/ready\": dial tcp 10.133.0.108:15021: connect: connection refused" start-of-body=
Oct 30 22:00:34.128866 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:34.128835    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd" podUID="5f3ddcc7-e832-42e9-97d4-1541f5a09044" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.108:15021/healthz/ready\": dial tcp 10.133.0.108:15021: connect: connection refused"
Oct 30 22:00:34.294492 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:34.294375    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8" podUID="84329f90-dc35-41dc-b593-12c556e80f73" containerName="main" probeResult="failure" output="Get \"https://10.133.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:00:34.295517 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:34.295491    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8"
Oct 30 22:00:34.460260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:34.454765    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.109:15021/healthz/ready\": dial tcp 10.133.0.109:15021: connect: connection refused" start-of-body=
Oct 30 22:00:34.460260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:34.454828    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq" podUID="84b9ee78-064c-4893-b81f-b9013b323448" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.109:15021/healthz/ready\": dial tcp 10.133.0.109:15021: connect: connection refused"
Oct 30 22:00:34.475259 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:34.473324    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj" podUID="93c5282f-81b6-4518-b673-68f527fdfd04" containerName="main" probeResult="failure" output="Get \"https://10.133.0.104:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:00:34.643197 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:34.643161    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj"
Oct 30 22:00:34.676398 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:34.674546    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-4c260505-65bb6f8647-p5g2x"
Oct 30 22:00:34.676398 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:34.674636    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-5tn88"
Oct 30 22:00:34.757260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:34.748978    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8"
Oct 30 22:00:35.041321 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:00:35.041184    4871 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33d005ac_b847_4fd5_af8f_5b0162b56610.slice\": RecentStats: unable to find data in memory cache]"
Oct 30 22:00:35.114147 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:35.114099    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.108:15021/healthz/ready\": dial tcp 10.133.0.108:15021: connect: connection refused" start-of-body=
Oct 30 22:00:35.114465 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:35.114414    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd" podUID="5f3ddcc7-e832-42e9-97d4-1541f5a09044" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.108:15021/healthz/ready\": dial tcp 10.133.0.108:15021: connect: connection refused"
Oct 30 22:00:35.441556 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:35.441477    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.109:15021/healthz/ready\": dial tcp 10.133.0.109:15021: connect: connection refused" start-of-body=
Oct 30 22:00:35.441751 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:35.441606    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq" podUID="84b9ee78-064c-4893-b81f-b9013b323448" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.109:15021/healthz/ready\": dial tcp 10.133.0.109:15021: connect: connection refused"
Oct 30 22:00:35.584387 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:00:35.584346    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:00:35.967102 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:35.967055    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:36.315147 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:00:36.315066    4871 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33d005ac_b847_4fd5_af8f_5b0162b56610.slice\": RecentStats: unable to find data in memory cache]"
Oct 30 22:00:36.637277 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:36.637228    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:37.131269 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:37.130355    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.108:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:00:37.131269 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:37.130442    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd" podUID="5f3ddcc7-e832-42e9-97d4-1541f5a09044" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.108:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:00:37.294616 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:37.294561    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:37.329396 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:37.329359    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"
Oct 30 22:00:37.330167 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:37.330105    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"
Oct 30 22:00:37.332175 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:37.332072    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"
Oct 30 22:00:37.481469 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:37.481361    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.109:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:00:37.481469 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:37.481422    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq" podUID="84b9ee78-064c-4893-b81f-b9013b323448" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.109:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:00:37.635498 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:37.635463    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"
Oct 30 22:00:37.635985 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:37.635962    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"
Oct 30 22:00:37.637084 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:37.637060    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"
Oct 30 22:00:38.228022 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:38.227992    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:41.411056 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:41.411009    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:46.085321 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:46.085232    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:51.212198 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:51.212107    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:51.389032 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:51.388993    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:51.552400 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:51.552315    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:56.134620 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:56.134568    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:59.603545 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:59.603500    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"]
Oct 30 22:00:59.609082 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:59.609060    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"
Oct 30 22:00:59.658556 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:59.658526    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-26743a-openshift-default-dockercfg-lddtx\""
Oct 30 22:00:59.710981 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:59.710942    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"]
Oct 30 22:00:59.731759 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:59.731727    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a43db63c-f279-45cb-9096-22b48545b36a-istiod-ca-cert\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x\" (UID: \"a43db63c-f279-45cb-9096-22b48545b36a\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"
Oct 30 22:00:59.731949 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:59.731796    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a43db63c-f279-45cb-9096-22b48545b36a-istio-podinfo\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x\" (UID: \"a43db63c-f279-45cb-9096-22b48545b36a\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"
Oct 30 22:00:59.731949 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:59.731903    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a43db63c-f279-45cb-9096-22b48545b36a-workload-socket\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x\" (UID: \"a43db63c-f279-45cb-9096-22b48545b36a\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"
Oct 30 22:00:59.732134 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:59.732115    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a43db63c-f279-45cb-9096-22b48545b36a-istio-envoy\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x\" (UID: \"a43db63c-f279-45cb-9096-22b48545b36a\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"
Oct 30 22:00:59.732200 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:59.732144    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5w5q\" (UniqueName: \"kubernetes.io/projected/a43db63c-f279-45cb-9096-22b48545b36a-kube-api-access-r5w5q\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x\" (UID: \"a43db63c-f279-45cb-9096-22b48545b36a\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"
Oct 30 22:00:59.732200 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:59.732177    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a43db63c-f279-45cb-9096-22b48545b36a-credential-socket\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x\" (UID: \"a43db63c-f279-45cb-9096-22b48545b36a\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"
Oct 30 22:00:59.732345 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:59.732326    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a43db63c-f279-45cb-9096-22b48545b36a-istio-token\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x\" (UID: \"a43db63c-f279-45cb-9096-22b48545b36a\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"
Oct 30 22:00:59.732537 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:59.732380    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a43db63c-f279-45cb-9096-22b48545b36a-workload-certs\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x\" (UID: \"a43db63c-f279-45cb-9096-22b48545b36a\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"
Oct 30 22:00:59.732537 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:59.732464    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a43db63c-f279-45cb-9096-22b48545b36a-istio-data\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x\" (UID: \"a43db63c-f279-45cb-9096-22b48545b36a\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"
Oct 30 22:00:59.833320 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:59.833280    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a43db63c-f279-45cb-9096-22b48545b36a-istio-envoy\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x\" (UID: \"a43db63c-f279-45cb-9096-22b48545b36a\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"
Oct 30 22:00:59.833532 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:59.833331    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5w5q\" (UniqueName: \"kubernetes.io/projected/a43db63c-f279-45cb-9096-22b48545b36a-kube-api-access-r5w5q\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x\" (UID: \"a43db63c-f279-45cb-9096-22b48545b36a\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"
Oct 30 22:00:59.833532 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:59.833366    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a43db63c-f279-45cb-9096-22b48545b36a-credential-socket\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x\" (UID: \"a43db63c-f279-45cb-9096-22b48545b36a\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"
Oct 30 22:00:59.833532 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:59.833395    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a43db63c-f279-45cb-9096-22b48545b36a-istio-token\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x\" (UID: \"a43db63c-f279-45cb-9096-22b48545b36a\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"
Oct 30 22:00:59.833698 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:59.833597    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a43db63c-f279-45cb-9096-22b48545b36a-workload-certs\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x\" (UID: \"a43db63c-f279-45cb-9096-22b48545b36a\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"
Oct 30 22:00:59.833698 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:59.833686    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a43db63c-f279-45cb-9096-22b48545b36a-istio-data\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x\" (UID: \"a43db63c-f279-45cb-9096-22b48545b36a\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"
Oct 30 22:00:59.833805 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:59.833742    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a43db63c-f279-45cb-9096-22b48545b36a-istiod-ca-cert\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x\" (UID: \"a43db63c-f279-45cb-9096-22b48545b36a\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"
Oct 30 22:00:59.833805 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:59.833784    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a43db63c-f279-45cb-9096-22b48545b36a-credential-socket\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x\" (UID: \"a43db63c-f279-45cb-9096-22b48545b36a\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"
Oct 30 22:00:59.833919 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:59.833856    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a43db63c-f279-45cb-9096-22b48545b36a-istio-podinfo\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x\" (UID: \"a43db63c-f279-45cb-9096-22b48545b36a\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"
Oct 30 22:00:59.833919 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:59.833887    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a43db63c-f279-45cb-9096-22b48545b36a-workload-socket\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x\" (UID: \"a43db63c-f279-45cb-9096-22b48545b36a\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"
Oct 30 22:00:59.834024 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:59.833984    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a43db63c-f279-45cb-9096-22b48545b36a-workload-certs\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x\" (UID: \"a43db63c-f279-45cb-9096-22b48545b36a\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"
Oct 30 22:00:59.834080 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:59.834047    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a43db63c-f279-45cb-9096-22b48545b36a-istio-data\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x\" (UID: \"a43db63c-f279-45cb-9096-22b48545b36a\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"
Oct 30 22:00:59.834286 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:59.834229    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a43db63c-f279-45cb-9096-22b48545b36a-workload-socket\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x\" (UID: \"a43db63c-f279-45cb-9096-22b48545b36a\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"
Oct 30 22:00:59.834410 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:59.834385    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a43db63c-f279-45cb-9096-22b48545b36a-istiod-ca-cert\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x\" (UID: \"a43db63c-f279-45cb-9096-22b48545b36a\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"
Oct 30 22:00:59.835745 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:59.835723    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a43db63c-f279-45cb-9096-22b48545b36a-istio-envoy\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x\" (UID: \"a43db63c-f279-45cb-9096-22b48545b36a\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"
Oct 30 22:00:59.836204 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:59.836181    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a43db63c-f279-45cb-9096-22b48545b36a-istio-podinfo\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x\" (UID: \"a43db63c-f279-45cb-9096-22b48545b36a\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"
Oct 30 22:00:59.892771 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:59.892738    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a43db63c-f279-45cb-9096-22b48545b36a-istio-token\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x\" (UID: \"a43db63c-f279-45cb-9096-22b48545b36a\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"
Oct 30 22:00:59.892958 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:59.892823    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5w5q\" (UniqueName: \"kubernetes.io/projected/a43db63c-f279-45cb-9096-22b48545b36a-kube-api-access-r5w5q\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x\" (UID: \"a43db63c-f279-45cb-9096-22b48545b36a\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"
Oct 30 22:00:59.924146 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:00:59.924111    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"
Oct 30 22:01:00.358002 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:00.357968    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"]
Oct 30 22:01:00.359270 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:01:00.359215    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda43db63c_f279_45cb_9096_22b48545b36a.slice/crio-0492f6519bb515a454ce83d190b76435e45d9b16a12b1bf7e986d1860e6da8b3 WatchSource:0}: Error finding container 0492f6519bb515a454ce83d190b76435e45d9b16a12b1bf7e986d1860e6da8b3: Status 404 returned error can't find the container with id 0492f6519bb515a454ce83d190b76435e45d9b16a12b1bf7e986d1860e6da8b3
Oct 30 22:01:00.361809 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:00.361776    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:01:00.361890 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:00.361851    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:01:00.361890 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:00.361881    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:01:00.605610 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:00.605499    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x" event={"ID":"a43db63c-f279-45cb-9096-22b48545b36a","Type":"ContainerStarted","Data":"462cf8e9e0e84e730a315ff3d76e2d608c2f769bbfcfdb9624069be66290ff53"}
Oct 30 22:01:00.605610 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:00.605577    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x" event={"ID":"a43db63c-f279-45cb-9096-22b48545b36a","Type":"ContainerStarted","Data":"0492f6519bb515a454ce83d190b76435e45d9b16a12b1bf7e986d1860e6da8b3"}
Oct 30 22:01:00.724009 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:00.723943    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x" podStartSLOduration=1.723927692 podStartE2EDuration="1.723927692s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:00:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:01:00.721417904 +0000 UTC m=+1402.898906520" watchObservedRunningTime="2026-10-30 22:01:00.723927692 +0000 UTC m=+1402.901416307"
Oct 30 22:01:00.762952 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:00.762907    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:00.924766 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:00.924724    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"
Oct 30 22:01:01.927845 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:01.927801    4871 patch_prober.go:29] interesting pod/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.110:15021/healthz/ready\": dial tcp 10.133.0.110:15021: connect: connection refused" start-of-body=
Oct 30 22:01:01.928377 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:01.928001    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x" podUID="a43db63c-f279-45cb-9096-22b48545b36a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.110:15021/healthz/ready\": dial tcp 10.133.0.110:15021: connect: connection refused"
Oct 30 22:01:02.927693 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:02.927654    4871 patch_prober.go:29] interesting pod/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.110:15021/healthz/ready\": dial tcp 10.133.0.110:15021: connect: connection refused" start-of-body=
Oct 30 22:01:02.927858 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:02.927714    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x" podUID="a43db63c-f279-45cb-9096-22b48545b36a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.110:15021/healthz/ready\": dial tcp 10.133.0.110:15021: connect: connection refused"
Oct 30 22:01:03.933731 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:03.933683    4871 patch_prober.go:29] interesting pod/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.110:15021/healthz/ready\": dial tcp 10.133.0.110:15021: connect: connection refused" start-of-body=
Oct 30 22:01:03.934308 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:03.933750    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x" podUID="a43db63c-f279-45cb-9096-22b48545b36a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.110:15021/healthz/ready\": dial tcp 10.133.0.110:15021: connect: connection refused"
Oct 30 22:01:04.929692 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:04.929650    4871 patch_prober.go:29] interesting pod/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.110:15021/healthz/ready\": dial tcp 10.133.0.110:15021: connect: connection refused" start-of-body=
Oct 30 22:01:04.929984 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:04.929955    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x" podUID="a43db63c-f279-45cb-9096-22b48545b36a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.110:15021/healthz/ready\": dial tcp 10.133.0.110:15021: connect: connection refused"
Oct 30 22:01:05.477152 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:05.477118    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:05.649228 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:01:05.649164    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:01:05.932261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:05.929733    4871 patch_prober.go:29] interesting pod/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.110:15021/healthz/ready\": dial tcp 10.133.0.110:15021: connect: connection refused" start-of-body=
Oct 30 22:01:05.932261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:05.929850    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x" podUID="a43db63c-f279-45cb-9096-22b48545b36a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.110:15021/healthz/ready\": dial tcp 10.133.0.110:15021: connect: connection refused"
Oct 30 22:01:06.924838 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:06.924796    4871 patch_prober.go:29] interesting pod/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.110:15021/healthz/ready\": dial tcp 10.133.0.110:15021: connect: connection refused" start-of-body=
Oct 30 22:01:06.925404 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:06.924865    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x" podUID="a43db63c-f279-45cb-9096-22b48545b36a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.110:15021/healthz/ready\": dial tcp 10.133.0.110:15021: connect: connection refused"
Oct 30 22:01:07.929753 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:07.929722    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"
Oct 30 22:01:07.930189 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:07.930060    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"
Oct 30 22:01:07.930887 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:07.930866    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"
Oct 30 22:01:10.434400 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:10.434356    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:13.135126 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:13.135077    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"]
Oct 30 22:01:13.140233 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:13.140213    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"
Oct 30 22:01:13.157441 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:13.157415    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8937fa65-1fb5-4614-9be1-9075e2b3f46b-istio-envoy\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52\" (UID: \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"
Oct 30 22:01:13.157649 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:13.157490    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8937fa65-1fb5-4614-9be1-9075e2b3f46b-workload-socket\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52\" (UID: \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"
Oct 30 22:01:13.157649 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:13.157622    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8937fa65-1fb5-4614-9be1-9075e2b3f46b-istio-data\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52\" (UID: \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"
Oct 30 22:01:13.157769 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:13.157746    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8937fa65-1fb5-4614-9be1-9075e2b3f46b-workload-certs\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52\" (UID: \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"
Oct 30 22:01:13.157810 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:13.157797    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8937fa65-1fb5-4614-9be1-9075e2b3f46b-istio-token\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52\" (UID: \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"
Oct 30 22:01:13.157864 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:13.157848    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbjfh\" (UniqueName: \"kubernetes.io/projected/8937fa65-1fb5-4614-9be1-9075e2b3f46b-kube-api-access-kbjfh\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52\" (UID: \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"
Oct 30 22:01:13.157977 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:13.157961    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8937fa65-1fb5-4614-9be1-9075e2b3f46b-istio-podinfo\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52\" (UID: \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"
Oct 30 22:01:13.158052 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:13.158035    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8937fa65-1fb5-4614-9be1-9075e2b3f46b-credential-socket\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52\" (UID: \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"
Oct 30 22:01:13.158112 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:13.158095    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8937fa65-1fb5-4614-9be1-9075e2b3f46b-istiod-ca-cert\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52\" (UID: \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"
Oct 30 22:01:13.251110 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:13.251071    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"]
Oct 30 22:01:13.259809 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:13.259779    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8937fa65-1fb5-4614-9be1-9075e2b3f46b-istio-podinfo\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52\" (UID: \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"
Oct 30 22:01:13.260573 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:13.260530    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8937fa65-1fb5-4614-9be1-9075e2b3f46b-credential-socket\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52\" (UID: \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"
Oct 30 22:01:13.260573 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:13.260570    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8937fa65-1fb5-4614-9be1-9075e2b3f46b-istiod-ca-cert\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52\" (UID: \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"
Oct 30 22:01:13.260845 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:13.260669    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8937fa65-1fb5-4614-9be1-9075e2b3f46b-istio-envoy\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52\" (UID: \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"
Oct 30 22:01:13.260845 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:13.260745    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8937fa65-1fb5-4614-9be1-9075e2b3f46b-workload-socket\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52\" (UID: \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"
Oct 30 22:01:13.260845 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:13.260799    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8937fa65-1fb5-4614-9be1-9075e2b3f46b-istio-data\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52\" (UID: \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"
Oct 30 22:01:13.261058 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:13.260879    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8937fa65-1fb5-4614-9be1-9075e2b3f46b-workload-certs\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52\" (UID: \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"
Oct 30 22:01:13.261058 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:13.260914    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8937fa65-1fb5-4614-9be1-9075e2b3f46b-istio-token\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52\" (UID: \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"
Oct 30 22:01:13.261284 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:13.260965    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbjfh\" (UniqueName: \"kubernetes.io/projected/8937fa65-1fb5-4614-9be1-9075e2b3f46b-kube-api-access-kbjfh\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52\" (UID: \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"
Oct 30 22:01:13.261827 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:13.261804    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8937fa65-1fb5-4614-9be1-9075e2b3f46b-workload-socket\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52\" (UID: \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"
Oct 30 22:01:13.262212 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:13.262188    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8937fa65-1fb5-4614-9be1-9075e2b3f46b-credential-socket\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52\" (UID: \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"
Oct 30 22:01:13.262359 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:13.262188    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8937fa65-1fb5-4614-9be1-9075e2b3f46b-istio-data\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52\" (UID: \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"
Oct 30 22:01:13.262724 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:13.262660    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8937fa65-1fb5-4614-9be1-9075e2b3f46b-workload-certs\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52\" (UID: \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"
Oct 30 22:01:13.262949 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:13.262929    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8937fa65-1fb5-4614-9be1-9075e2b3f46b-istiod-ca-cert\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52\" (UID: \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"
Oct 30 22:01:13.265177 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:13.265156    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8937fa65-1fb5-4614-9be1-9075e2b3f46b-istio-envoy\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52\" (UID: \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"
Oct 30 22:01:13.273162 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:13.273138    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8937fa65-1fb5-4614-9be1-9075e2b3f46b-istio-podinfo\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52\" (UID: \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"
Oct 30 22:01:13.330504 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:13.330451    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbjfh\" (UniqueName: \"kubernetes.io/projected/8937fa65-1fb5-4614-9be1-9075e2b3f46b-kube-api-access-kbjfh\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52\" (UID: \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"
Oct 30 22:01:13.330752 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:13.330606    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8937fa65-1fb5-4614-9be1-9075e2b3f46b-istio-token\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52\" (UID: \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"
Oct 30 22:01:13.455975 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:13.455890    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"
Oct 30 22:01:13.943353 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:13.937121    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"]
Oct 30 22:01:13.953224 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:01:13.953190    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8937fa65_1fb5_4614_9be1_9075e2b3f46b.slice/crio-e08e7b9d37e6a8a5c164a13ef3c4e55cde40efdeb7c03c22f93f01e9b139ff41 WatchSource:0}: Error finding container e08e7b9d37e6a8a5c164a13ef3c4e55cde40efdeb7c03c22f93f01e9b139ff41: Status 404 returned error can't find the container with id e08e7b9d37e6a8a5c164a13ef3c4e55cde40efdeb7c03c22f93f01e9b139ff41
Oct 30 22:01:13.955687 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:13.955652    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:01:13.955788 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:13.955718    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:01:13.955788 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:13.955746    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:01:14.767003 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:14.766968    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52" event={"ID":"8937fa65-1fb5-4614-9be1-9075e2b3f46b","Type":"ContainerStarted","Data":"9c8cd1f4cefef255025a31f10f048eba8e711ea542dc7f633f3497981b512888"}
Oct 30 22:01:14.767525 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:14.767030    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52" event={"ID":"8937fa65-1fb5-4614-9be1-9075e2b3f46b","Type":"ContainerStarted","Data":"e08e7b9d37e6a8a5c164a13ef3c4e55cde40efdeb7c03c22f93f01e9b139ff41"}
Oct 30 22:01:14.886286 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:14.886203    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52" podStartSLOduration=1.886187061 podStartE2EDuration="1.886187061s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:01:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:01:14.884191009 +0000 UTC m=+1417.061679626" watchObservedRunningTime="2026-10-30 22:01:14.886187061 +0000 UTC m=+1417.063675677"
Oct 30 22:01:15.325318 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:15.325200    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:15.460266 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:15.460164    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"
Oct 30 22:01:15.460476 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:15.460339    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused" start-of-body=
Oct 30 22:01:15.460476 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:15.460428    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52" podUID="8937fa65-1fb5-4614-9be1-9075e2b3f46b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused"
Oct 30 22:01:16.457035 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:16.456998    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused" start-of-body=
Oct 30 22:01:16.457512 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:16.457070    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52" podUID="8937fa65-1fb5-4614-9be1-9075e2b3f46b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused"
Oct 30 22:01:17.019346 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:17.019183    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:17.468385 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:17.466397    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused" start-of-body=
Oct 30 22:01:17.468385 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:17.466460    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52" podUID="8937fa65-1fb5-4614-9be1-9075e2b3f46b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused"
Oct 30 22:01:18.457023 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:18.456977    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused" start-of-body=
Oct 30 22:01:18.457209 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:18.457047    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52" podUID="8937fa65-1fb5-4614-9be1-9075e2b3f46b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused"
Oct 30 22:01:19.456868 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:19.456824    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused" start-of-body=
Oct 30 22:01:19.457319 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:19.456898    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52" podUID="8937fa65-1fb5-4614-9be1-9075e2b3f46b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused"
Oct 30 22:01:20.367814 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:20.367771    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:20.463262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:20.459643    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused" start-of-body=
Oct 30 22:01:20.463262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:20.459712    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52" podUID="8937fa65-1fb5-4614-9be1-9075e2b3f46b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused"
Oct 30 22:01:21.383136 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:21.383083    4871 patch_prober.go:29] interesting pod/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused" start-of-body=
Oct 30 22:01:21.383422 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:21.383154    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz" podUID="198884dd-8c10-4d1f-9f64-6e00c0e7b6ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused"
Oct 30 22:01:21.765343 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:21.760104    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"
Oct 30 22:01:21.765343 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:21.761456    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"
Oct 30 22:01:21.765343 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:21.763734    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"
Oct 30 22:01:21.866684 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:21.866649    4871 generic.go:356] "Generic (PLEG): container finished" podID="198884dd-8c10-4d1f-9f64-6e00c0e7b6ab" containerID="b419997ba915969d7decd8f7a38ecdfe34dca49729e2b8747e3e95f3ea373138" exitCode=0
Oct 30 22:01:21.866890 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:21.866740    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz" event={"ID":"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab","Type":"ContainerDied","Data":"b419997ba915969d7decd8f7a38ecdfe34dca49729e2b8747e3e95f3ea373138"}
Oct 30 22:01:21.867211 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:21.867191    4871 scope.go:122] "RemoveContainer" containerID="b419997ba915969d7decd8f7a38ecdfe34dca49729e2b8747e3e95f3ea373138"
Oct 30 22:01:21.868549 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:21.868522    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:01:21.868625 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:21.868589    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:01:21.868625 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:21.868621    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:01:22.883966 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:22.883170    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz" event={"ID":"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab","Type":"ContainerStarted","Data":"baf0dc7131c0f6a4930be0190234056d6f1deaa5ff25fd7e2f2312b18e3406e4"}
Oct 30 22:01:23.186566 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:23.186460    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"
Oct 30 22:01:23.186753 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:23.186719    4871 patch_prober.go:29] interesting pod/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused" start-of-body=
Oct 30 22:01:23.186819 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:23.186772    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz" podUID="198884dd-8c10-4d1f-9f64-6e00c0e7b6ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused"
Oct 30 22:01:24.183495 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:24.183453    4871 patch_prober.go:29] interesting pod/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused" start-of-body=
Oct 30 22:01:24.183940 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:24.183527    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz" podUID="198884dd-8c10-4d1f-9f64-6e00c0e7b6ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused"
Oct 30 22:01:25.183386 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:25.183349    4871 patch_prober.go:29] interesting pod/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused" start-of-body=
Oct 30 22:01:25.183575 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:25.183404    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz" podUID="198884dd-8c10-4d1f-9f64-6e00c0e7b6ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused"
Oct 30 22:01:25.274069 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:25.274026    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:26.183653 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:26.183614    4871 patch_prober.go:29] interesting pod/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused" start-of-body=
Oct 30 22:01:26.184084 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:26.183674    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz" podUID="198884dd-8c10-4d1f-9f64-6e00c0e7b6ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused"
Oct 30 22:01:27.183605 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:27.183567    4871 patch_prober.go:29] interesting pod/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused" start-of-body=
Oct 30 22:01:27.183796 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:27.183626    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz" podUID="198884dd-8c10-4d1f-9f64-6e00c0e7b6ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused"
Oct 30 22:01:28.183042 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:28.182996    4871 patch_prober.go:29] interesting pod/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused" start-of-body=
Oct 30 22:01:28.183246 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:28.183061    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz" podUID="198884dd-8c10-4d1f-9f64-6e00c0e7b6ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused"
Oct 30 22:01:29.235818 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:29.235780    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"
Oct 30 22:01:29.237157 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:29.236621    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"
Oct 30 22:01:29.238452 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:29.238429    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"
Oct 30 22:01:31.633819 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:31.633777    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:35.033868 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:35.033823    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:35.451962 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:35.451901    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:35.571928 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:01:35.571894    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:01:36.318462 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:36.318426    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:40.030628 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:40.030572    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:43.681912 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:43.681850    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:44.624403 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:44.624340    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:46.233127 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:46.233088    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:46.401844 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:46.401793    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:49.288263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:49.282231    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:54.103671 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:54.103629    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:54.777344 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:54.777297    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:55.510591 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:55.510548    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:58.609404 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:01:58.609358    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:01.312953 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:01.312911    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:01.926171 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:01.926131    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"]
Oct 30 22:02:01.932221 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:01.932187    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"
Oct 30 22:02:02.087295 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:02.068804    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-695f5c-openshift-default-dockercfg-gspj9\""
Oct 30 22:02:02.136289 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:02.114051    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:02.171857 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:02.171820    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"]
Oct 30 22:02:02.216484 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:02.216371    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48a6bcac-9719-4358-9019-71fd0a388ba1-istio-data\") pod \"e2e-api-a-695f5c-openshift-default-94db46fbf-9c272\" (UID: \"48a6bcac-9719-4358-9019-71fd0a388ba1\") " pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"
Oct 30 22:02:02.216736 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:02.216705    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48a6bcac-9719-4358-9019-71fd0a388ba1-istiod-ca-cert\") pod \"e2e-api-a-695f5c-openshift-default-94db46fbf-9c272\" (UID: \"48a6bcac-9719-4358-9019-71fd0a388ba1\") " pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"
Oct 30 22:02:02.216893 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:02.216870    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48a6bcac-9719-4358-9019-71fd0a388ba1-istio-podinfo\") pod \"e2e-api-a-695f5c-openshift-default-94db46fbf-9c272\" (UID: \"48a6bcac-9719-4358-9019-71fd0a388ba1\") " pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"
Oct 30 22:02:02.217074 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:02.217059    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48a6bcac-9719-4358-9019-71fd0a388ba1-workload-socket\") pod \"e2e-api-a-695f5c-openshift-default-94db46fbf-9c272\" (UID: \"48a6bcac-9719-4358-9019-71fd0a388ba1\") " pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"
Oct 30 22:02:02.217198 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:02.217183    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48a6bcac-9719-4358-9019-71fd0a388ba1-credential-socket\") pod \"e2e-api-a-695f5c-openshift-default-94db46fbf-9c272\" (UID: \"48a6bcac-9719-4358-9019-71fd0a388ba1\") " pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"
Oct 30 22:02:02.217342 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:02.217325    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48a6bcac-9719-4358-9019-71fd0a388ba1-istio-token\") pod \"e2e-api-a-695f5c-openshift-default-94db46fbf-9c272\" (UID: \"48a6bcac-9719-4358-9019-71fd0a388ba1\") " pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"
Oct 30 22:02:02.230414 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:02.230372    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xk8p\" (UniqueName: \"kubernetes.io/projected/48a6bcac-9719-4358-9019-71fd0a388ba1-kube-api-access-5xk8p\") pod \"e2e-api-a-695f5c-openshift-default-94db46fbf-9c272\" (UID: \"48a6bcac-9719-4358-9019-71fd0a388ba1\") " pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"
Oct 30 22:02:02.230711 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:02.230693    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48a6bcac-9719-4358-9019-71fd0a388ba1-workload-certs\") pod \"e2e-api-a-695f5c-openshift-default-94db46fbf-9c272\" (UID: \"48a6bcac-9719-4358-9019-71fd0a388ba1\") " pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"
Oct 30 22:02:02.230865 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:02.230851    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48a6bcac-9719-4358-9019-71fd0a388ba1-istio-envoy\") pod \"e2e-api-a-695f5c-openshift-default-94db46fbf-9c272\" (UID: \"48a6bcac-9719-4358-9019-71fd0a388ba1\") " pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"
Oct 30 22:02:02.293794 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:02.293758    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:02.332857 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:02.332813    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48a6bcac-9719-4358-9019-71fd0a388ba1-istio-data\") pod \"e2e-api-a-695f5c-openshift-default-94db46fbf-9c272\" (UID: \"48a6bcac-9719-4358-9019-71fd0a388ba1\") " pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"
Oct 30 22:02:02.333532 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:02.333475    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48a6bcac-9719-4358-9019-71fd0a388ba1-istiod-ca-cert\") pod \"e2e-api-a-695f5c-openshift-default-94db46fbf-9c272\" (UID: \"48a6bcac-9719-4358-9019-71fd0a388ba1\") " pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"
Oct 30 22:02:02.333883 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:02.333848    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48a6bcac-9719-4358-9019-71fd0a388ba1-istio-podinfo\") pod \"e2e-api-a-695f5c-openshift-default-94db46fbf-9c272\" (UID: \"48a6bcac-9719-4358-9019-71fd0a388ba1\") " pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"
Oct 30 22:02:02.334127 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:02.334105    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48a6bcac-9719-4358-9019-71fd0a388ba1-workload-socket\") pod \"e2e-api-a-695f5c-openshift-default-94db46fbf-9c272\" (UID: \"48a6bcac-9719-4358-9019-71fd0a388ba1\") " pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"
Oct 30 22:02:02.350699 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:02.350659    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48a6bcac-9719-4358-9019-71fd0a388ba1-credential-socket\") pod \"e2e-api-a-695f5c-openshift-default-94db46fbf-9c272\" (UID: \"48a6bcac-9719-4358-9019-71fd0a388ba1\") " pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"
Oct 30 22:02:02.351038 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:02.351014    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48a6bcac-9719-4358-9019-71fd0a388ba1-istio-token\") pod \"e2e-api-a-695f5c-openshift-default-94db46fbf-9c272\" (UID: \"48a6bcac-9719-4358-9019-71fd0a388ba1\") " pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"
Oct 30 22:02:02.351174 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:02.351157    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xk8p\" (UniqueName: \"kubernetes.io/projected/48a6bcac-9719-4358-9019-71fd0a388ba1-kube-api-access-5xk8p\") pod \"e2e-api-a-695f5c-openshift-default-94db46fbf-9c272\" (UID: \"48a6bcac-9719-4358-9019-71fd0a388ba1\") " pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"
Oct 30 22:02:02.351805 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:02.351780    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48a6bcac-9719-4358-9019-71fd0a388ba1-credential-socket\") pod \"e2e-api-a-695f5c-openshift-default-94db46fbf-9c272\" (UID: \"48a6bcac-9719-4358-9019-71fd0a388ba1\") " pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"
Oct 30 22:02:02.354572 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:02.352502    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48a6bcac-9719-4358-9019-71fd0a388ba1-istio-data\") pod \"e2e-api-a-695f5c-openshift-default-94db46fbf-9c272\" (UID: \"48a6bcac-9719-4358-9019-71fd0a388ba1\") " pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"
Oct 30 22:02:02.354572 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:02.353136    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48a6bcac-9719-4358-9019-71fd0a388ba1-istiod-ca-cert\") pod \"e2e-api-a-695f5c-openshift-default-94db46fbf-9c272\" (UID: \"48a6bcac-9719-4358-9019-71fd0a388ba1\") " pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"
Oct 30 22:02:02.354955 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:02.354934    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48a6bcac-9719-4358-9019-71fd0a388ba1-workload-certs\") pod \"e2e-api-a-695f5c-openshift-default-94db46fbf-9c272\" (UID: \"48a6bcac-9719-4358-9019-71fd0a388ba1\") " pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"
Oct 30 22:02:02.372894 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:02.355518    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48a6bcac-9719-4358-9019-71fd0a388ba1-workload-certs\") pod \"e2e-api-a-695f5c-openshift-default-94db46fbf-9c272\" (UID: \"48a6bcac-9719-4358-9019-71fd0a388ba1\") " pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"
Oct 30 22:02:02.372894 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:02.355776    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48a6bcac-9719-4358-9019-71fd0a388ba1-workload-socket\") pod \"e2e-api-a-695f5c-openshift-default-94db46fbf-9c272\" (UID: \"48a6bcac-9719-4358-9019-71fd0a388ba1\") " pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"
Oct 30 22:02:02.373650 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:02.363357    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48a6bcac-9719-4358-9019-71fd0a388ba1-istio-envoy\") pod \"e2e-api-a-695f5c-openshift-default-94db46fbf-9c272\" (UID: \"48a6bcac-9719-4358-9019-71fd0a388ba1\") " pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"
Oct 30 22:02:02.398967 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:02.398902    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48a6bcac-9719-4358-9019-71fd0a388ba1-istio-envoy\") pod \"e2e-api-a-695f5c-openshift-default-94db46fbf-9c272\" (UID: \"48a6bcac-9719-4358-9019-71fd0a388ba1\") " pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"
Oct 30 22:02:02.399820 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:02.399761    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48a6bcac-9719-4358-9019-71fd0a388ba1-istio-podinfo\") pod \"e2e-api-a-695f5c-openshift-default-94db46fbf-9c272\" (UID: \"48a6bcac-9719-4358-9019-71fd0a388ba1\") " pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"
Oct 30 22:02:02.452126 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:02.452025    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48a6bcac-9719-4358-9019-71fd0a388ba1-istio-token\") pod \"e2e-api-a-695f5c-openshift-default-94db46fbf-9c272\" (UID: \"48a6bcac-9719-4358-9019-71fd0a388ba1\") " pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"
Oct 30 22:02:02.462385 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:02.462130    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xk8p\" (UniqueName: \"kubernetes.io/projected/48a6bcac-9719-4358-9019-71fd0a388ba1-kube-api-access-5xk8p\") pod \"e2e-api-a-695f5c-openshift-default-94db46fbf-9c272\" (UID: \"48a6bcac-9719-4358-9019-71fd0a388ba1\") " pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"
Oct 30 22:02:02.637826 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:02.637773    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"
Oct 30 22:02:03.077374 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:03.077293    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"]
Oct 30 22:02:03.110046 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:02:03.110012    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48a6bcac_9719_4358_9019_71fd0a388ba1.slice/crio-a658cca2940a68835afc76ca8042275fb0f5d155bb294a25a103f112b1ad18bd WatchSource:0}: Error finding container a658cca2940a68835afc76ca8042275fb0f5d155bb294a25a103f112b1ad18bd: Status 404 returned error can't find the container with id a658cca2940a68835afc76ca8042275fb0f5d155bb294a25a103f112b1ad18bd
Oct 30 22:02:03.112495 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:03.112459    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:02:03.112615 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:03.112565    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:02:03.112680 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:03.112615    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:02:03.317067 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:03.316995    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:03.471223 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:03.469971    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272" event={"ID":"48a6bcac-9719-4358-9019-71fd0a388ba1","Type":"ContainerStarted","Data":"fa7716b450593c2f234b240a01087cdb064d22e8b477c568e49d862e0549c63f"}
Oct 30 22:02:03.471223 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:03.470044    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272" event={"ID":"48a6bcac-9719-4358-9019-71fd0a388ba1","Type":"ContainerStarted","Data":"a658cca2940a68835afc76ca8042275fb0f5d155bb294a25a103f112b1ad18bd"}
Oct 30 22:02:03.587561 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:03.587502    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272" podStartSLOduration=2.587487943 podStartE2EDuration="2.587487943s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:02:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:02:03.585026856 +0000 UTC m=+1465.762515474" watchObservedRunningTime="2026-10-30 22:02:03.587487943 +0000 UTC m=+1465.764976559"
Oct 30 22:02:03.638357 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:03.638314    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"
Oct 30 22:02:04.644882 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:04.644848    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"
Oct 30 22:02:05.485846 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:05.485816    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"
Oct 30 22:02:05.487185 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:05.487163    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"
Oct 30 22:02:05.572208 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:02:05.572168    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:02:06.395578 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:06.395540    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:06.608748 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:06.608714    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:06.847319 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:06.847204    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:07.013062 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:07.013028    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:08.177361 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:08.177325    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:08.773869 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:08.773832    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:09.670032 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:09.669995    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:09.893218 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:09.893181    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:13.023922 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:13.023885    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:13.250074 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:13.250038    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:13.607434 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:13.607399    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"]
Oct 30 22:02:13.612215 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:13.612190    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"
Oct 30 22:02:13.674177 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:13.674145    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-2d2f63-openshift-default-dockercfg-gzggm\""
Oct 30 22:02:13.715268 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:13.715213    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:13.823989 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:13.823950    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"]
Oct 30 22:02:13.826724 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:13.826667    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef03dee1-e693-42a0-ace8-4a28e099b92b-workload-certs\") pod \"e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr\" (UID: \"ef03dee1-e693-42a0-ace8-4a28e099b92b\") " pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"
Oct 30 22:02:13.828939 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:13.828902    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gkvb\" (UniqueName: \"kubernetes.io/projected/ef03dee1-e693-42a0-ace8-4a28e099b92b-kube-api-access-7gkvb\") pod \"e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr\" (UID: \"ef03dee1-e693-42a0-ace8-4a28e099b92b\") " pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"
Oct 30 22:02:13.829257 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:13.829221    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef03dee1-e693-42a0-ace8-4a28e099b92b-istio-envoy\") pod \"e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr\" (UID: \"ef03dee1-e693-42a0-ace8-4a28e099b92b\") " pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"
Oct 30 22:02:13.829607 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:13.829588    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef03dee1-e693-42a0-ace8-4a28e099b92b-credential-socket\") pod \"e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr\" (UID: \"ef03dee1-e693-42a0-ace8-4a28e099b92b\") " pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"
Oct 30 22:02:13.829773 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:13.829757    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef03dee1-e693-42a0-ace8-4a28e099b92b-istio-podinfo\") pod \"e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr\" (UID: \"ef03dee1-e693-42a0-ace8-4a28e099b92b\") " pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"
Oct 30 22:02:13.830293 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:13.830273    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef03dee1-e693-42a0-ace8-4a28e099b92b-istio-token\") pod \"e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr\" (UID: \"ef03dee1-e693-42a0-ace8-4a28e099b92b\") " pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"
Oct 30 22:02:13.830766 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:13.830747    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef03dee1-e693-42a0-ace8-4a28e099b92b-workload-socket\") pod \"e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr\" (UID: \"ef03dee1-e693-42a0-ace8-4a28e099b92b\") " pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"
Oct 30 22:02:13.830928 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:13.830911    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef03dee1-e693-42a0-ace8-4a28e099b92b-istiod-ca-cert\") pod \"e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr\" (UID: \"ef03dee1-e693-42a0-ace8-4a28e099b92b\") " pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"
Oct 30 22:02:13.831095 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:13.831079    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef03dee1-e693-42a0-ace8-4a28e099b92b-istio-data\") pod \"e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr\" (UID: \"ef03dee1-e693-42a0-ace8-4a28e099b92b\") " pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"
Oct 30 22:02:14.091140 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:14.091101    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:14.092510 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:14.092440    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef03dee1-e693-42a0-ace8-4a28e099b92b-workload-certs\") pod \"e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr\" (UID: \"ef03dee1-e693-42a0-ace8-4a28e099b92b\") " pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"
Oct 30 22:02:14.092738 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:14.092715    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gkvb\" (UniqueName: \"kubernetes.io/projected/ef03dee1-e693-42a0-ace8-4a28e099b92b-kube-api-access-7gkvb\") pod \"e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr\" (UID: \"ef03dee1-e693-42a0-ace8-4a28e099b92b\") " pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"
Oct 30 22:02:14.092963 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:14.092946    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef03dee1-e693-42a0-ace8-4a28e099b92b-istio-envoy\") pod \"e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr\" (UID: \"ef03dee1-e693-42a0-ace8-4a28e099b92b\") " pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"
Oct 30 22:02:14.093302 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:14.093203    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef03dee1-e693-42a0-ace8-4a28e099b92b-credential-socket\") pod \"e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr\" (UID: \"ef03dee1-e693-42a0-ace8-4a28e099b92b\") " pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"
Oct 30 22:02:14.093444 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:14.093426    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef03dee1-e693-42a0-ace8-4a28e099b92b-istio-podinfo\") pod \"e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr\" (UID: \"ef03dee1-e693-42a0-ace8-4a28e099b92b\") " pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"
Oct 30 22:02:14.093627 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:14.093611    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef03dee1-e693-42a0-ace8-4a28e099b92b-istio-token\") pod \"e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr\" (UID: \"ef03dee1-e693-42a0-ace8-4a28e099b92b\") " pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"
Oct 30 22:02:14.093846 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:14.093829    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef03dee1-e693-42a0-ace8-4a28e099b92b-workload-socket\") pod \"e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr\" (UID: \"ef03dee1-e693-42a0-ace8-4a28e099b92b\") " pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"
Oct 30 22:02:14.093965 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:14.093949    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef03dee1-e693-42a0-ace8-4a28e099b92b-istiod-ca-cert\") pod \"e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr\" (UID: \"ef03dee1-e693-42a0-ace8-4a28e099b92b\") " pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"
Oct 30 22:02:14.094077 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:14.094064    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef03dee1-e693-42a0-ace8-4a28e099b92b-istio-data\") pod \"e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr\" (UID: \"ef03dee1-e693-42a0-ace8-4a28e099b92b\") " pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"
Oct 30 22:02:14.094848 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:14.094822    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef03dee1-e693-42a0-ace8-4a28e099b92b-istio-data\") pod \"e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr\" (UID: \"ef03dee1-e693-42a0-ace8-4a28e099b92b\") " pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"
Oct 30 22:02:14.095567 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:14.095545    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef03dee1-e693-42a0-ace8-4a28e099b92b-workload-certs\") pod \"e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr\" (UID: \"ef03dee1-e693-42a0-ace8-4a28e099b92b\") " pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"
Oct 30 22:02:14.124822 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:14.124773    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef03dee1-e693-42a0-ace8-4a28e099b92b-credential-socket\") pod \"e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr\" (UID: \"ef03dee1-e693-42a0-ace8-4a28e099b92b\") " pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"
Oct 30 22:02:14.129261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:14.126915    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef03dee1-e693-42a0-ace8-4a28e099b92b-istiod-ca-cert\") pod \"e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr\" (UID: \"ef03dee1-e693-42a0-ace8-4a28e099b92b\") " pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"
Oct 30 22:02:14.129261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:14.127935    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef03dee1-e693-42a0-ace8-4a28e099b92b-istio-envoy\") pod \"e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr\" (UID: \"ef03dee1-e693-42a0-ace8-4a28e099b92b\") " pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"
Oct 30 22:02:14.142336 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:14.141870    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef03dee1-e693-42a0-ace8-4a28e099b92b-workload-socket\") pod \"e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr\" (UID: \"ef03dee1-e693-42a0-ace8-4a28e099b92b\") " pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"
Oct 30 22:02:14.154031 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:14.153994    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef03dee1-e693-42a0-ace8-4a28e099b92b-istio-podinfo\") pod \"e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr\" (UID: \"ef03dee1-e693-42a0-ace8-4a28e099b92b\") " pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"
Oct 30 22:02:14.162107 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:14.162064    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gkvb\" (UniqueName: \"kubernetes.io/projected/ef03dee1-e693-42a0-ace8-4a28e099b92b-kube-api-access-7gkvb\") pod \"e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr\" (UID: \"ef03dee1-e693-42a0-ace8-4a28e099b92b\") " pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"
Oct 30 22:02:14.247525 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:14.247004    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef03dee1-e693-42a0-ace8-4a28e099b92b-istio-token\") pod \"e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr\" (UID: \"ef03dee1-e693-42a0-ace8-4a28e099b92b\") " pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"
Oct 30 22:02:14.535613 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:14.535575    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"
Oct 30 22:02:14.974002 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:02:14.973961    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef03dee1_e693_42a0_ace8_4a28e099b92b.slice/crio-0897c2a8d63467761ecd998f506bdc8eb3c4108531eabb83d21a4bf13cf743eb WatchSource:0}: Error finding container 0897c2a8d63467761ecd998f506bdc8eb3c4108531eabb83d21a4bf13cf743eb: Status 404 returned error can't find the container with id 0897c2a8d63467761ecd998f506bdc8eb3c4108531eabb83d21a4bf13cf743eb
Oct 30 22:02:14.974174 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:14.974064    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"]
Oct 30 22:02:14.977113 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:14.977081    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:02:14.977215 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:14.977153    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:02:14.977215 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:14.977207    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:02:15.580097 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:15.580057    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr" event={"ID":"ef03dee1-e693-42a0-ace8-4a28e099b92b","Type":"ContainerStarted","Data":"2b49035c6e6dbaec7dd1491af63a36a98ff5b497551fe9664f34702e21c712c9"}
Oct 30 22:02:15.580531 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:15.580129    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr" event={"ID":"ef03dee1-e693-42a0-ace8-4a28e099b92b","Type":"ContainerStarted","Data":"0897c2a8d63467761ecd998f506bdc8eb3c4108531eabb83d21a4bf13cf743eb"}
Oct 30 22:02:15.706770 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:15.706711    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr" podStartSLOduration=2.706695996 podStartE2EDuration="2.706695996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:02:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:02:15.705276429 +0000 UTC m=+1477.882765049" watchObservedRunningTime="2026-10-30 22:02:15.706695996 +0000 UTC m=+1477.884184612"
Oct 30 22:02:15.944890 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:15.944834    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:16.536480 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:16.536434    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"
Oct 30 22:02:16.543896 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:16.543867    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"
Oct 30 22:02:16.587748 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:16.587710    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"
Oct 30 22:02:16.588679 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:16.588656    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"
Oct 30 22:02:18.086834 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:18.086751    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:18.409533 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:18.409471    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:18.609168 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:18.609125    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:22.901822 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:22.901780    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:23.276853 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:23.276755    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:25.104859 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:25.104427    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:25.433332 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:25.426689    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:25.732331 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:25.727805    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-btfp8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:02:25.894854 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:25.894805    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8" podUID="ffc32abd-61d0-447a-9050-2ad518ca585f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:02:26.541392 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:26.541350    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-bpm9h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:02:26.542068 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:26.542030    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h" podUID="b28e1c99-b48d-4a4e-a225-ebba779327e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:02:26.606376 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:26.606335    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-mrq5j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:02:26.606667 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:26.606632    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j" podUID="19911666-f25a-4dd7-9789-7b9374b61cfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:02:28.687133 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:28.687100    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:33.246643 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:33.246606    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:35.568955 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:02:35.568916    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:02:36.657939 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:36.657905    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:37.808317 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:37.808273    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:38.202573 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:38.202545    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx_8d401654-8a4c-4a79-b2d3-612765312421/extract/0.log"
Oct 30 22:02:38.209351 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:38.209323    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx_8d401654-8a4c-4a79-b2d3-612765312421/extract/0.log"
Oct 30 22:02:38.215581 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:38.215557    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/2.log"
Oct 30 22:02:38.222633 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:38.222606    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/2.log"
Oct 30 22:02:38.223432 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:38.223410    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzntd_2360fddc-52a5-42a5-82ca-a05aff84d217/ovn-acl-logging/0.log"
Oct 30 22:02:38.230393 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:38.230372    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzntd_2360fddc-52a5-42a5-82ca-a05aff84d217/ovn-acl-logging/0.log"
Oct 30 22:02:42.064783 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:42.064750    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-695f5c-744b8495dc-smjpj"]
Oct 30 22:02:42.069388 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:42.069365    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-695f5c-744b8495dc-smjpj"
Oct 30 22:02:42.119554 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:42.119512    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-695f5c-serving-cert\""
Oct 30 22:02:42.148550 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:42.148511    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ea1b9a4-cdb1-475d-82f4-7b286e8410dc-openshift-service-ca\") pod \"maas-api-e2e-api-a-695f5c-744b8495dc-smjpj\" (UID: \"9ea1b9a4-cdb1-475d-82f4-7b286e8410dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-695f5c-744b8495dc-smjpj"
Oct 30 22:02:42.148798 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:42.148766    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ea1b9a4-cdb1-475d-82f4-7b286e8410dc-metrics-certs\") pod \"maas-api-e2e-api-a-695f5c-744b8495dc-smjpj\" (UID: \"9ea1b9a4-cdb1-475d-82f4-7b286e8410dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-695f5c-744b8495dc-smjpj"
Oct 30 22:02:42.148881 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:42.148863    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ea1b9a4-cdb1-475d-82f4-7b286e8410dc-maas-api-tls\") pod \"maas-api-e2e-api-a-695f5c-744b8495dc-smjpj\" (UID: \"9ea1b9a4-cdb1-475d-82f4-7b286e8410dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-695f5c-744b8495dc-smjpj"
Oct 30 22:02:42.148976 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:42.148945    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42z79\" (UniqueName: \"kubernetes.io/projected/9ea1b9a4-cdb1-475d-82f4-7b286e8410dc-kube-api-access-42z79\") pod \"maas-api-e2e-api-a-695f5c-744b8495dc-smjpj\" (UID: \"9ea1b9a4-cdb1-475d-82f4-7b286e8410dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-695f5c-744b8495dc-smjpj"
Oct 30 22:02:42.173715 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:42.173676    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-695f5c-744b8495dc-smjpj"]
Oct 30 22:02:42.250643 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:42.250601    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ea1b9a4-cdb1-475d-82f4-7b286e8410dc-metrics-certs\") pod \"maas-api-e2e-api-a-695f5c-744b8495dc-smjpj\" (UID: \"9ea1b9a4-cdb1-475d-82f4-7b286e8410dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-695f5c-744b8495dc-smjpj"
Oct 30 22:02:42.250855 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:42.250665    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ea1b9a4-cdb1-475d-82f4-7b286e8410dc-maas-api-tls\") pod \"maas-api-e2e-api-a-695f5c-744b8495dc-smjpj\" (UID: \"9ea1b9a4-cdb1-475d-82f4-7b286e8410dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-695f5c-744b8495dc-smjpj"
Oct 30 22:02:42.250855 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:42.250688    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42z79\" (UniqueName: \"kubernetes.io/projected/9ea1b9a4-cdb1-475d-82f4-7b286e8410dc-kube-api-access-42z79\") pod \"maas-api-e2e-api-a-695f5c-744b8495dc-smjpj\" (UID: \"9ea1b9a4-cdb1-475d-82f4-7b286e8410dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-695f5c-744b8495dc-smjpj"
Oct 30 22:02:42.250855 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:42.250806    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ea1b9a4-cdb1-475d-82f4-7b286e8410dc-openshift-service-ca\") pod \"maas-api-e2e-api-a-695f5c-744b8495dc-smjpj\" (UID: \"9ea1b9a4-cdb1-475d-82f4-7b286e8410dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-695f5c-744b8495dc-smjpj"
Oct 30 22:02:42.250855 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:02:42.250817    4871 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-695f5c-serving-cert: secret "maas-api-e2e-api-a-695f5c-serving-cert" not found
Oct 30 22:02:42.251077 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:02:42.250887    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ea1b9a4-cdb1-475d-82f4-7b286e8410dc-maas-api-tls podName:9ea1b9a4-cdb1-475d-82f4-7b286e8410dc nodeName:}" failed. No retries permitted until 2026-10-30 22:02:42.750862563 +0000 UTC m=+1504.928351172 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9ea1b9a4-cdb1-475d-82f4-7b286e8410dc-maas-api-tls") pod "maas-api-e2e-api-a-695f5c-744b8495dc-smjpj" (UID: "9ea1b9a4-cdb1-475d-82f4-7b286e8410dc") : secret "maas-api-e2e-api-a-695f5c-serving-cert" not found
Oct 30 22:02:42.251518 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:42.251495    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ea1b9a4-cdb1-475d-82f4-7b286e8410dc-openshift-service-ca\") pod \"maas-api-e2e-api-a-695f5c-744b8495dc-smjpj\" (UID: \"9ea1b9a4-cdb1-475d-82f4-7b286e8410dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-695f5c-744b8495dc-smjpj"
Oct 30 22:02:42.253321 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:42.253301    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ea1b9a4-cdb1-475d-82f4-7b286e8410dc-metrics-certs\") pod \"maas-api-e2e-api-a-695f5c-744b8495dc-smjpj\" (UID: \"9ea1b9a4-cdb1-475d-82f4-7b286e8410dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-695f5c-744b8495dc-smjpj"
Oct 30 22:02:42.333363 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:42.333170    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:42.342648 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:42.339917    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-42z79\" (UniqueName: \"kubernetes.io/projected/9ea1b9a4-cdb1-475d-82f4-7b286e8410dc-kube-api-access-42z79\") pod \"maas-api-e2e-api-a-695f5c-744b8495dc-smjpj\" (UID: \"9ea1b9a4-cdb1-475d-82f4-7b286e8410dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-695f5c-744b8495dc-smjpj"
Oct 30 22:02:42.488401 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:42.488329    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:42.646717 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:42.646665    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:42.774735 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:42.774693    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ea1b9a4-cdb1-475d-82f4-7b286e8410dc-maas-api-tls\") pod \"maas-api-e2e-api-a-695f5c-744b8495dc-smjpj\" (UID: \"9ea1b9a4-cdb1-475d-82f4-7b286e8410dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-695f5c-744b8495dc-smjpj"
Oct 30 22:02:42.777261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:42.777218    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ea1b9a4-cdb1-475d-82f4-7b286e8410dc-maas-api-tls\") pod \"maas-api-e2e-api-a-695f5c-744b8495dc-smjpj\" (UID: \"9ea1b9a4-cdb1-475d-82f4-7b286e8410dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-695f5c-744b8495dc-smjpj"
Oct 30 22:02:42.983305 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:42.983215    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-695f5c-744b8495dc-smjpj"
Oct 30 22:02:43.365696 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:43.365661    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-695f5c-744b8495dc-smjpj"]
Oct 30 22:02:43.367519 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:02:43.367482    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ea1b9a4_cdb1_475d_82f4_7b286e8410dc.slice/crio-504e70587186f0ef498cb24181d111d1883592e7d5037ba4562906f6a989f9ed WatchSource:0}: Error finding container 504e70587186f0ef498cb24181d111d1883592e7d5037ba4562906f6a989f9ed: Status 404 returned error can't find the container with id 504e70587186f0ef498cb24181d111d1883592e7d5037ba4562906f6a989f9ed
Oct 30 22:02:43.368893 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:43.368876    4871 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:02:44.254495 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:44.254452    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-695f5c-744b8495dc-smjpj" event={"ID":"9ea1b9a4-cdb1-475d-82f4-7b286e8410dc","Type":"ContainerStarted","Data":"504e70587186f0ef498cb24181d111d1883592e7d5037ba4562906f6a989f9ed"}
Oct 30 22:02:45.264247 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:45.264212    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-695f5c-744b8495dc-smjpj" event={"ID":"9ea1b9a4-cdb1-475d-82f4-7b286e8410dc","Type":"ContainerStarted","Data":"d12ee8e00618ec10b114ead08dc778eaf4d3ae2a17fb892d08e60a098853232f"}
Oct 30 22:02:45.264640 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:45.264308    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-695f5c-744b8495dc-smjpj"
Oct 30 22:02:45.384144 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:45.384083    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-695f5c-744b8495dc-smjpj" podStartSLOduration=2.46250329 podStartE2EDuration="3.384068097s" totalImagesPullingTime="921.564807ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:02:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:02:45.382876902 +0000 UTC m=+1507.560365518" watchObservedRunningTime="2026-10-30 22:02:45.384068097 +0000 UTC m=+1507.561556728"
Oct 30 22:02:47.162585 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:47.162551    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:51.278908 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:51.278876    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-695f5c-744b8495dc-smjpj"
Oct 30 22:02:51.737691 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:51.737652    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:55.961973 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:55.961943    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk"]
Oct 30 22:02:55.966902 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:55.966872    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk"
Oct 30 22:02:56.012155 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:56.012100    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zdqk\" (UniqueName: \"kubernetes.io/projected/f61805de-c5e9-42ef-9382-1e1c5c7d188e-kube-api-access-5zdqk\") pod \"maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk\" (UID: \"f61805de-c5e9-42ef-9382-1e1c5c7d188e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk"
Oct 30 22:02:56.012391 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:56.012220    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f61805de-c5e9-42ef-9382-1e1c5c7d188e-metrics-certs\") pod \"maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk\" (UID: \"f61805de-c5e9-42ef-9382-1e1c5c7d188e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk"
Oct 30 22:02:56.012391 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:56.012367    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f61805de-c5e9-42ef-9382-1e1c5c7d188e-maas-api-tls\") pod \"maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk\" (UID: \"f61805de-c5e9-42ef-9382-1e1c5c7d188e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk"
Oct 30 22:02:56.012529 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:56.012515    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f61805de-c5e9-42ef-9382-1e1c5c7d188e-openshift-service-ca\") pod \"maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk\" (UID: \"f61805de-c5e9-42ef-9382-1e1c5c7d188e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk"
Oct 30 22:02:56.014675 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:56.014637    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk"]
Oct 30 22:02:56.055839 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:56.055802    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-2d2f63-serving-cert\""
Oct 30 22:02:56.114220 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:56.114182    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f61805de-c5e9-42ef-9382-1e1c5c7d188e-metrics-certs\") pod \"maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk\" (UID: \"f61805de-c5e9-42ef-9382-1e1c5c7d188e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk"
Oct 30 22:02:56.114422 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:56.114324    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f61805de-c5e9-42ef-9382-1e1c5c7d188e-maas-api-tls\") pod \"maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk\" (UID: \"f61805de-c5e9-42ef-9382-1e1c5c7d188e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk"
Oct 30 22:02:56.114478 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:56.114459    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f61805de-c5e9-42ef-9382-1e1c5c7d188e-openshift-service-ca\") pod \"maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk\" (UID: \"f61805de-c5e9-42ef-9382-1e1c5c7d188e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk"
Oct 30 22:02:56.114520 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:02:56.114506    4871 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-2d2f63-serving-cert: secret "maas-api-e2e-api-a-2d2f63-serving-cert" not found
Oct 30 22:02:56.114567 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:56.114532    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zdqk\" (UniqueName: \"kubernetes.io/projected/f61805de-c5e9-42ef-9382-1e1c5c7d188e-kube-api-access-5zdqk\") pod \"maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk\" (UID: \"f61805de-c5e9-42ef-9382-1e1c5c7d188e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk"
Oct 30 22:02:56.114609 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:02:56.114578    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f61805de-c5e9-42ef-9382-1e1c5c7d188e-maas-api-tls podName:f61805de-c5e9-42ef-9382-1e1c5c7d188e nodeName:}" failed. No retries permitted until 2026-10-30 22:02:56.614561452 +0000 UTC m=+1518.792050051 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f61805de-c5e9-42ef-9382-1e1c5c7d188e-maas-api-tls") pod "maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk" (UID: "f61805de-c5e9-42ef-9382-1e1c5c7d188e") : secret "maas-api-e2e-api-a-2d2f63-serving-cert" not found
Oct 30 22:02:56.115168 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:56.115146    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f61805de-c5e9-42ef-9382-1e1c5c7d188e-openshift-service-ca\") pod \"maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk\" (UID: \"f61805de-c5e9-42ef-9382-1e1c5c7d188e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk"
Oct 30 22:02:56.116899 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:56.116870    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f61805de-c5e9-42ef-9382-1e1c5c7d188e-metrics-certs\") pod \"maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk\" (UID: \"f61805de-c5e9-42ef-9382-1e1c5c7d188e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk"
Oct 30 22:02:56.144292 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:56.144220    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:56.176103 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:56.175864    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zdqk\" (UniqueName: \"kubernetes.io/projected/f61805de-c5e9-42ef-9382-1e1c5c7d188e-kube-api-access-5zdqk\") pod \"maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk\" (UID: \"f61805de-c5e9-42ef-9382-1e1c5c7d188e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk"
Oct 30 22:02:56.313338 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:56.310911    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:56.535849 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:56.535811    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:56.644562 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:56.644525    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f61805de-c5e9-42ef-9382-1e1c5c7d188e-maas-api-tls\") pod \"maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk\" (UID: \"f61805de-c5e9-42ef-9382-1e1c5c7d188e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk"
Oct 30 22:02:56.647031 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:56.647006    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f61805de-c5e9-42ef-9382-1e1c5c7d188e-maas-api-tls\") pod \"maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk\" (UID: \"f61805de-c5e9-42ef-9382-1e1c5c7d188e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk"
Oct 30 22:02:56.882849 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:56.882808    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk"
Oct 30 22:02:57.266447 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:57.266412    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk"]
Oct 30 22:02:57.268692 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:02:57.268660    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf61805de_c5e9_42ef_9382_1e1c5c7d188e.slice/crio-94843748c19afe0e4c51c9d088fa8dca109c876c35d720a47fdd42e62dcb8574 WatchSource:0}: Error finding container 94843748c19afe0e4c51c9d088fa8dca109c876c35d720a47fdd42e62dcb8574: Status 404 returned error can't find the container with id 94843748c19afe0e4c51c9d088fa8dca109c876c35d720a47fdd42e62dcb8574
Oct 30 22:02:57.467839 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:57.467798    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk" event={"ID":"f61805de-c5e9-42ef-9382-1e1c5c7d188e","Type":"ContainerStarted","Data":"94843748c19afe0e4c51c9d088fa8dca109c876c35d720a47fdd42e62dcb8574"}
Oct 30 22:02:58.477527 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:58.477423    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk" event={"ID":"f61805de-c5e9-42ef-9382-1e1c5c7d188e","Type":"ContainerStarted","Data":"4b5fbe8f2698d37894f279abdaab680d9b5e6225f7d1679f2d824535b05dc556"}
Oct 30 22:02:58.477912 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:58.477545    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk"
Oct 30 22:02:58.595964 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:02:58.595903    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk" podStartSLOduration=2.698669462 podStartE2EDuration="3.595889072s" totalImagesPullingTime="897.21961ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:02:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:02:58.589954757 +0000 UTC m=+1520.767443374" watchObservedRunningTime="2026-10-30 22:02:58.595889072 +0000 UTC m=+1520.773377689"
Oct 30 22:03:00.884429 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:00.884387    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:04.492540 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:04.492510    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk"
Oct 30 22:03:05.469644 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:05.469553    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:05.664696 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:03:05.663345    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:03:10.333265 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:10.330817    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:10.600642 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:10.600546    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:11.190322 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:11.190286    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"]
Oct 30 22:03:11.218538 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:11.218511    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"
Oct 30 22:03:11.325935 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:11.325894    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:11.328868 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:11.328838    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-istio-token\") pod \"e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm\" (UID: \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\") " pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"
Oct 30 22:03:11.359719 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:11.359673    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zqnl\" (UniqueName: \"kubernetes.io/projected/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-kube-api-access-7zqnl\") pod \"e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm\" (UID: \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\") " pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"
Oct 30 22:03:11.364486 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:11.364459    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-istio-envoy\") pod \"e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm\" (UID: \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\") " pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"
Oct 30 22:03:11.364773 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:11.364740    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-istio-data\") pod \"e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm\" (UID: \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\") " pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"
Oct 30 22:03:11.368006 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:11.365168    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-workload-certs\") pod \"e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm\" (UID: \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\") " pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"
Oct 30 22:03:11.368301 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:11.368276    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-credential-socket\") pod \"e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm\" (UID: \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\") " pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"
Oct 30 22:03:11.461512 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:11.461423    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-istiod-ca-cert\") pod \"e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm\" (UID: \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\") " pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"
Oct 30 22:03:11.461985 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:11.461961    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-istio-podinfo\") pod \"e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm\" (UID: \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\") " pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"
Oct 30 22:03:11.462196 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:11.462178    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-workload-socket\") pod \"e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm\" (UID: \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\") " pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"
Oct 30 22:03:11.462483 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:11.355717    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-f9b7e7-openshift-default-dockercfg-7nstc\""
Oct 30 22:03:11.462606 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:11.364351    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"]
Oct 30 22:03:11.602266 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:11.590631    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-workload-certs\") pod \"e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm\" (UID: \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\") " pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"
Oct 30 22:03:11.602266 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:11.590739    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-credential-socket\") pod \"e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm\" (UID: \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\") " pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"
Oct 30 22:03:11.602266 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:11.590771    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-istiod-ca-cert\") pod \"e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm\" (UID: \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\") " pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"
Oct 30 22:03:11.602266 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:11.590859    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-istio-podinfo\") pod \"e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm\" (UID: \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\") " pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"
Oct 30 22:03:11.602266 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:11.590910    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-workload-socket\") pod \"e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm\" (UID: \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\") " pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"
Oct 30 22:03:11.602266 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:11.591002    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-istio-token\") pod \"e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm\" (UID: \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\") " pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"
Oct 30 22:03:11.602266 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:11.591029    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zqnl\" (UniqueName: \"kubernetes.io/projected/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-kube-api-access-7zqnl\") pod \"e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm\" (UID: \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\") " pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"
Oct 30 22:03:11.602266 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:11.591091    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-istio-envoy\") pod \"e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm\" (UID: \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\") " pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"
Oct 30 22:03:11.602266 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:11.591115    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-istio-data\") pod \"e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm\" (UID: \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\") " pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"
Oct 30 22:03:11.632260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:11.621391    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-workload-socket\") pod \"e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm\" (UID: \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\") " pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"
Oct 30 22:03:11.632260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:11.623203    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-istio-data\") pod \"e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm\" (UID: \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\") " pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"
Oct 30 22:03:11.659264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:11.642840    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-credential-socket\") pod \"e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm\" (UID: \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\") " pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"
Oct 30 22:03:11.659264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:11.643102    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-workload-certs\") pod \"e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm\" (UID: \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\") " pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"
Oct 30 22:03:11.659264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:11.643816    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-istiod-ca-cert\") pod \"e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm\" (UID: \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\") " pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"
Oct 30 22:03:11.933433 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:11.790150    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-istio-podinfo\") pod \"e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm\" (UID: \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\") " pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"
Oct 30 22:03:11.933433 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:11.843524    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:11.985269 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:11.973642    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zqnl\" (UniqueName: \"kubernetes.io/projected/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-kube-api-access-7zqnl\") pod \"e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm\" (UID: \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\") " pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"
Oct 30 22:03:12.017553 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:12.015475    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-istio-envoy\") pod \"e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm\" (UID: \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\") " pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"
Oct 30 22:03:12.059260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:12.049525    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-istio-token\") pod \"e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm\" (UID: \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\") " pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"
Oct 30 22:03:12.168767 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:12.168732    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"
Oct 30 22:03:12.634960 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:12.634906    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"]
Oct 30 22:03:12.658897 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:03:12.658858    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0cbb7ad_f150_4562_bb4c_ad9dce4545bd.slice/crio-0c920744aff9b96e6daaa5a41a472ef73531a4eb4127c7c9adcd2928dbc16ca2 WatchSource:0}: Error finding container 0c920744aff9b96e6daaa5a41a472ef73531a4eb4127c7c9adcd2928dbc16ca2: Status 404 returned error can't find the container with id 0c920744aff9b96e6daaa5a41a472ef73531a4eb4127c7c9adcd2928dbc16ca2
Oct 30 22:03:12.662293 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:12.662260    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:03:12.662364 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:12.662348    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:03:12.662403 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:12.662389    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:03:13.077147 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:13.077107    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm" event={"ID":"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd","Type":"ContainerStarted","Data":"d0b4495eb4008b7ed008ce1ded98730c0ab8dca8bdcb2dd342e6f7eed2fed0f3"}
Oct 30 22:03:13.077344 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:13.077173    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm" event={"ID":"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd","Type":"ContainerStarted","Data":"0c920744aff9b96e6daaa5a41a472ef73531a4eb4127c7c9adcd2928dbc16ca2"}
Oct 30 22:03:13.169478 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:13.169398    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"
Oct 30 22:03:13.174988 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:13.174961    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"
Oct 30 22:03:13.241420 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:13.241361    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm" podStartSLOduration=2.241344407 podStartE2EDuration="2.241344407s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:03:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:03:13.239525542 +0000 UTC m=+1535.417014160" watchObservedRunningTime="2026-10-30 22:03:13.241344407 +0000 UTC m=+1535.418833023"
Oct 30 22:03:14.087706 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:14.087675    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"
Oct 30 22:03:14.089084 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:14.089057    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"
Oct 30 22:03:15.395205 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:15.395165    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:16.011649 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:16.011616    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:16.213225 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:16.213194    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:16.376052 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:16.376010    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:17.123907 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:17.123873    4871 generic.go:356] "Generic (PLEG): container finished" podID="48a6bcac-9719-4358-9019-71fd0a388ba1" containerID="fa7716b450593c2f234b240a01087cdb064d22e8b477c568e49d862e0549c63f" exitCode=0
Oct 30 22:03:17.124394 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:17.123948    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272" event={"ID":"48a6bcac-9719-4358-9019-71fd0a388ba1","Type":"ContainerDied","Data":"fa7716b450593c2f234b240a01087cdb064d22e8b477c568e49d862e0549c63f"}
Oct 30 22:03:17.124462 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:17.124440    4871 scope.go:122] "RemoveContainer" containerID="fa7716b450593c2f234b240a01087cdb064d22e8b477c568e49d862e0549c63f"
Oct 30 22:03:17.125890 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:17.125863    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:03:17.125973 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:17.125927    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:03:17.125973 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:17.125957    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:03:18.141000 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:18.140961    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272" event={"ID":"48a6bcac-9719-4358-9019-71fd0a388ba1","Type":"ContainerStarted","Data":"9809017f93fd0d95d4b865dafea8cc1b1073da5111a4332a8c88bba6a6aac440"}
Oct 30 22:03:18.638617 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:18.638576    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"
Oct 30 22:03:18.640149 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:18.640118    4871 patch_prober.go:29] interesting pod/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused" start-of-body=
Oct 30 22:03:18.640294 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:18.640176    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272" podUID="48a6bcac-9719-4358-9019-71fd0a388ba1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused"
Oct 30 22:03:19.638152 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:19.638119    4871 patch_prober.go:29] interesting pod/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused" start-of-body=
Oct 30 22:03:19.638571 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:19.638170    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272" podUID="48a6bcac-9719-4358-9019-71fd0a388ba1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused"
Oct 30 22:03:19.752172 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:19.752126    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:20.024955 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:20.024871    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:20.447487 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:20.447384    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:20.486693 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:20.486663    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"
Oct 30 22:03:20.858682 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:20.847752    4871 patch_prober.go:29] interesting pod/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused" start-of-body=
Oct 30 22:03:20.859310 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:20.859266    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272" podUID="48a6bcac-9719-4358-9019-71fd0a388ba1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused"
Oct 30 22:03:21.638782 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:21.638741    4871 patch_prober.go:29] interesting pod/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused" start-of-body=
Oct 30 22:03:21.638995 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:21.638815    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272" podUID="48a6bcac-9719-4358-9019-71fd0a388ba1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused"
Oct 30 22:03:22.638402 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:22.638361    4871 patch_prober.go:29] interesting pod/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused" start-of-body=
Oct 30 22:03:22.638808 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:22.638438    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272" podUID="48a6bcac-9719-4358-9019-71fd0a388ba1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused"
Oct 30 22:03:23.638382 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:23.638341    4871 patch_prober.go:29] interesting pod/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused" start-of-body=
Oct 30 22:03:23.638584 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:23.638410    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272" podUID="48a6bcac-9719-4358-9019-71fd0a388ba1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused"
Oct 30 22:03:24.644344 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:24.644308    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"
Oct 30 22:03:24.645724 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:24.645701    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"
Oct 30 22:03:25.178075 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:25.178042    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:30.038137 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:30.038096    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:33.285260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:33.275308    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:34.064652 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:34.064614    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:34.713836 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:34.713800    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"]
Oct 30 22:03:34.718715 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:34.718691    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"
Oct 30 22:03:34.774734 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:34.774688    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-a5f7b9-openshift-default-dockercfg-8tmpg\""
Oct 30 22:03:34.786328 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:34.786289    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"]
Oct 30 22:03:34.809743 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:34.809704    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-istio-data\") pod \"e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp\" (UID: \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\") " pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"
Oct 30 22:03:34.809921 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:34.809760    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-workload-socket\") pod \"e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp\" (UID: \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\") " pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"
Oct 30 22:03:34.809968 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:34.809940    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-istio-envoy\") pod \"e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp\" (UID: \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\") " pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"
Oct 30 22:03:34.810028 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:34.809999    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-istiod-ca-cert\") pod \"e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp\" (UID: \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\") " pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"
Oct 30 22:03:34.810145 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:34.810125    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-workload-certs\") pod \"e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp\" (UID: \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\") " pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"
Oct 30 22:03:34.810214 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:34.810163    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-credential-socket\") pod \"e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp\" (UID: \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\") " pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"
Oct 30 22:03:34.810352 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:34.810335    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-istio-podinfo\") pod \"e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp\" (UID: \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\") " pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"
Oct 30 22:03:34.810418 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:34.810386    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d872l\" (UniqueName: \"kubernetes.io/projected/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-kube-api-access-d872l\") pod \"e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp\" (UID: \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\") " pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"
Oct 30 22:03:34.810539 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:34.810516    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-istio-token\") pod \"e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp\" (UID: \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\") " pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"
Oct 30 22:03:34.844401 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:34.844366    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:34.912918 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:34.912788    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-istio-podinfo\") pod \"e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp\" (UID: \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\") " pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"
Oct 30 22:03:34.913543 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:34.913509    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d872l\" (UniqueName: \"kubernetes.io/projected/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-kube-api-access-d872l\") pod \"e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp\" (UID: \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\") " pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"
Oct 30 22:03:34.914728 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:34.914705    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-istio-token\") pod \"e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp\" (UID: \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\") " pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"
Oct 30 22:03:34.915300 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:34.915280    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-istio-data\") pod \"e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp\" (UID: \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\") " pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"
Oct 30 22:03:34.915856 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:34.915834    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-workload-socket\") pod \"e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp\" (UID: \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\") " pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"
Oct 30 22:03:34.916563 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:34.916541    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-istio-envoy\") pod \"e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp\" (UID: \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\") " pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"
Oct 30 22:03:34.917075 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:34.915777    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-istio-data\") pod \"e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp\" (UID: \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\") " pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"
Oct 30 22:03:34.918017 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:34.916304    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-workload-socket\") pod \"e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp\" (UID: \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\") " pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"
Oct 30 22:03:34.918173 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:34.917934    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-istiod-ca-cert\") pod \"e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp\" (UID: \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\") " pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"
Oct 30 22:03:34.918395 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:34.918378    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-istiod-ca-cert\") pod \"e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp\" (UID: \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\") " pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"
Oct 30 22:03:34.918593 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:34.918578    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-workload-certs\") pod \"e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp\" (UID: \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\") " pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"
Oct 30 22:03:34.919020 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:34.919000    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-credential-socket\") pod \"e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp\" (UID: \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\") " pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"
Oct 30 22:03:34.920328 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:34.919449    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-credential-socket\") pod \"e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp\" (UID: \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\") " pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"
Oct 30 22:03:34.920527 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:34.918949    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-workload-certs\") pod \"e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp\" (UID: \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\") " pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"
Oct 30 22:03:34.920628 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:34.919676    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-istio-envoy\") pod \"e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp\" (UID: \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\") " pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"
Oct 30 22:03:34.920628 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:34.920089    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-istio-podinfo\") pod \"e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp\" (UID: \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\") " pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"
Oct 30 22:03:34.998320 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:34.995773    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-istio-token\") pod \"e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp\" (UID: \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\") " pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"
Oct 30 22:03:35.009271 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:35.007675    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d872l\" (UniqueName: \"kubernetes.io/projected/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-kube-api-access-d872l\") pod \"e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp\" (UID: \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\") " pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"
Oct 30 22:03:35.013260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:35.009755    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:35.047271 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:35.034539    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"
Oct 30 22:03:35.479137 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:35.479095    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:35.685369 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:03:35.684444    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:03:35.773275 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:35.773205    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"]
Oct 30 22:03:35.773752 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:03:35.773579    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode00cf9ef_6182_4fc8_b564_ea2425c9e5a5.slice/crio-5a0fa1c4eb7693da08eece46e0f12ac83b49c9fd40c3bd89d89eb23ed96c78d2 WatchSource:0}: Error finding container 5a0fa1c4eb7693da08eece46e0f12ac83b49c9fd40c3bd89d89eb23ed96c78d2: Status 404 returned error can't find the container with id 5a0fa1c4eb7693da08eece46e0f12ac83b49c9fd40c3bd89d89eb23ed96c78d2
Oct 30 22:03:35.776431 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:35.776400    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:03:35.776516 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:35.776466    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:03:35.776562 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:35.776515    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:03:36.637464 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:36.637416    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp" event={"ID":"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5","Type":"ContainerStarted","Data":"389157db519b317d7fd8a6435373d87d78a5106c90d871b73e23738271a5ab58"}
Oct 30 22:03:36.637650 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:36.637491    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp" event={"ID":"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5","Type":"ContainerStarted","Data":"5a0fa1c4eb7693da08eece46e0f12ac83b49c9fd40c3bd89d89eb23ed96c78d2"}
Oct 30 22:03:36.801809 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:36.801755    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp" podStartSLOduration=2.801740878 podStartE2EDuration="2.801740878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:03:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:03:36.799493828 +0000 UTC m=+1558.976982446" watchObservedRunningTime="2026-10-30 22:03:36.801740878 +0000 UTC m=+1558.979229495"
Oct 30 22:03:37.018319 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:37.017920    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:37.069854 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:37.069822    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"
Oct 30 22:03:37.080962 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:37.080928    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"
Oct 30 22:03:37.647054 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:37.647015    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"
Oct 30 22:03:37.648520 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:37.648492    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"
Oct 30 22:03:39.347101 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:39.347063    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:39.658046 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:39.658008    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:39.884939 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:39.884902    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:39.973722 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:39.973647    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"]
Oct 30 22:03:39.988512 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:39.982959    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"
Oct 30 22:03:40.071263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:40.055361    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-4cb675-openshift-default-dockercfg-56dbk\""
Oct 30 22:03:40.082758 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:40.082715    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5e633ba-5844-4f32-956d-8757ce76acce-istio-token\") pod \"e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l\" (UID: \"d5e633ba-5844-4f32-956d-8757ce76acce\") " pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"
Oct 30 22:03:40.083125 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:40.083084    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5e633ba-5844-4f32-956d-8757ce76acce-workload-socket\") pod \"e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l\" (UID: \"d5e633ba-5844-4f32-956d-8757ce76acce\") " pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"
Oct 30 22:03:40.083353 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:40.083319    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8nnq\" (UniqueName: \"kubernetes.io/projected/d5e633ba-5844-4f32-956d-8757ce76acce-kube-api-access-h8nnq\") pod \"e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l\" (UID: \"d5e633ba-5844-4f32-956d-8757ce76acce\") " pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"
Oct 30 22:03:40.083592 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:40.083572    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5e633ba-5844-4f32-956d-8757ce76acce-workload-certs\") pod \"e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l\" (UID: \"d5e633ba-5844-4f32-956d-8757ce76acce\") " pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"
Oct 30 22:03:40.083750 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:40.083733    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5e633ba-5844-4f32-956d-8757ce76acce-istiod-ca-cert\") pod \"e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l\" (UID: \"d5e633ba-5844-4f32-956d-8757ce76acce\") " pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"
Oct 30 22:03:40.083899 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:40.083883    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5e633ba-5844-4f32-956d-8757ce76acce-istio-data\") pod \"e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l\" (UID: \"d5e633ba-5844-4f32-956d-8757ce76acce\") " pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"
Oct 30 22:03:40.084196 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:40.084178    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5e633ba-5844-4f32-956d-8757ce76acce-istio-podinfo\") pod \"e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l\" (UID: \"d5e633ba-5844-4f32-956d-8757ce76acce\") " pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"
Oct 30 22:03:40.084425 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:40.084409    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5e633ba-5844-4f32-956d-8757ce76acce-istio-envoy\") pod \"e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l\" (UID: \"d5e633ba-5844-4f32-956d-8757ce76acce\") " pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"
Oct 30 22:03:40.084594 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:40.084564    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5e633ba-5844-4f32-956d-8757ce76acce-credential-socket\") pod \"e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l\" (UID: \"d5e633ba-5844-4f32-956d-8757ce76acce\") " pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"
Oct 30 22:03:40.260315 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:40.228214    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5e633ba-5844-4f32-956d-8757ce76acce-istio-envoy\") pod \"e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l\" (UID: \"d5e633ba-5844-4f32-956d-8757ce76acce\") " pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"
Oct 30 22:03:40.291171 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:40.291124    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5e633ba-5844-4f32-956d-8757ce76acce-credential-socket\") pod \"e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l\" (UID: \"d5e633ba-5844-4f32-956d-8757ce76acce\") " pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"
Oct 30 22:03:40.291526 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:40.291507    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5e633ba-5844-4f32-956d-8757ce76acce-istio-token\") pod \"e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l\" (UID: \"d5e633ba-5844-4f32-956d-8757ce76acce\") " pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"
Oct 30 22:03:40.291702 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:40.291688    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5e633ba-5844-4f32-956d-8757ce76acce-workload-socket\") pod \"e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l\" (UID: \"d5e633ba-5844-4f32-956d-8757ce76acce\") " pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"
Oct 30 22:03:40.291819 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:40.291805    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8nnq\" (UniqueName: \"kubernetes.io/projected/d5e633ba-5844-4f32-956d-8757ce76acce-kube-api-access-h8nnq\") pod \"e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l\" (UID: \"d5e633ba-5844-4f32-956d-8757ce76acce\") " pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"
Oct 30 22:03:40.291989 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:40.291975    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5e633ba-5844-4f32-956d-8757ce76acce-workload-certs\") pod \"e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l\" (UID: \"d5e633ba-5844-4f32-956d-8757ce76acce\") " pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"
Oct 30 22:03:40.292089 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:40.292076    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5e633ba-5844-4f32-956d-8757ce76acce-istiod-ca-cert\") pod \"e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l\" (UID: \"d5e633ba-5844-4f32-956d-8757ce76acce\") " pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"
Oct 30 22:03:40.292191 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:40.292177    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5e633ba-5844-4f32-956d-8757ce76acce-istio-data\") pod \"e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l\" (UID: \"d5e633ba-5844-4f32-956d-8757ce76acce\") " pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"
Oct 30 22:03:40.292824 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:40.292798    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5e633ba-5844-4f32-956d-8757ce76acce-workload-socket\") pod \"e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l\" (UID: \"d5e633ba-5844-4f32-956d-8757ce76acce\") " pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"
Oct 30 22:03:40.292983 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:40.292819    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5e633ba-5844-4f32-956d-8757ce76acce-istio-data\") pod \"e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l\" (UID: \"d5e633ba-5844-4f32-956d-8757ce76acce\") " pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"
Oct 30 22:03:40.329355 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:40.238517    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:40.341484 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:40.293071    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5e633ba-5844-4f32-956d-8757ce76acce-credential-socket\") pod \"e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l\" (UID: \"d5e633ba-5844-4f32-956d-8757ce76acce\") " pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"
Oct 30 22:03:40.351336 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:40.260800    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5e633ba-5844-4f32-956d-8757ce76acce-istio-envoy\") pod \"e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l\" (UID: \"d5e633ba-5844-4f32-956d-8757ce76acce\") " pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"
Oct 30 22:03:40.351994 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:40.293098    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5e633ba-5844-4f32-956d-8757ce76acce-istio-podinfo\") pod \"e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l\" (UID: \"d5e633ba-5844-4f32-956d-8757ce76acce\") " pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"
Oct 30 22:03:40.352127 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:40.303535    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5e633ba-5844-4f32-956d-8757ce76acce-istiod-ca-cert\") pod \"e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l\" (UID: \"d5e633ba-5844-4f32-956d-8757ce76acce\") " pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"
Oct 30 22:03:40.352217 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:40.303857    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5e633ba-5844-4f32-956d-8757ce76acce-workload-certs\") pod \"e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l\" (UID: \"d5e633ba-5844-4f32-956d-8757ce76acce\") " pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"
Oct 30 22:03:40.365795 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:40.365711    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5e633ba-5844-4f32-956d-8757ce76acce-istio-token\") pod \"e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l\" (UID: \"d5e633ba-5844-4f32-956d-8757ce76acce\") " pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"
Oct 30 22:03:40.369039 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:40.368959    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8nnq\" (UniqueName: \"kubernetes.io/projected/d5e633ba-5844-4f32-956d-8757ce76acce-kube-api-access-h8nnq\") pod \"e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l\" (UID: \"d5e633ba-5844-4f32-956d-8757ce76acce\") " pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"
Oct 30 22:03:40.455263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:40.450373    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"]
Oct 30 22:03:40.455263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:40.452897    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5e633ba-5844-4f32-956d-8757ce76acce-istio-podinfo\") pod \"e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l\" (UID: \"d5e633ba-5844-4f32-956d-8757ce76acce\") " pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"
Oct 30 22:03:40.535483 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:40.528083    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:40.564619 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:40.564571    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"
Oct 30 22:03:40.792448 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:40.792310    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:41.271566 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:41.271540    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"]
Oct 30 22:03:41.271729 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:03:41.271695    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5e633ba_5844_4f32_956d_8757ce76acce.slice/crio-e95f5103a65d9341c5f5d0a32af18d4c76eb016bb00147413544ecfaa533ee3c WatchSource:0}: Error finding container e95f5103a65d9341c5f5d0a32af18d4c76eb016bb00147413544ecfaa533ee3c: Status 404 returned error can't find the container with id e95f5103a65d9341c5f5d0a32af18d4c76eb016bb00147413544ecfaa533ee3c
Oct 30 22:03:41.274264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:41.274202    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:03:41.274375 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:41.274309    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:03:41.274375 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:41.274354    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:03:41.816490 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:41.816451    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l" event={"ID":"d5e633ba-5844-4f32-956d-8757ce76acce","Type":"ContainerStarted","Data":"be36c755fc6a62139fac349c74f708a0df0de657e27ccd05b45943253d316aae"}
Oct 30 22:03:41.816960 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:41.816516    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l" event={"ID":"d5e633ba-5844-4f32-956d-8757ce76acce","Type":"ContainerStarted","Data":"e95f5103a65d9341c5f5d0a32af18d4c76eb016bb00147413544ecfaa533ee3c"}
Oct 30 22:03:42.003523 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:42.003452    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l" podStartSLOduration=3.0034297 podStartE2EDuration="3.0034297s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:03:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:03:42.000314366 +0000 UTC m=+1564.177803010" watchObservedRunningTime="2026-10-30 22:03:42.0034297 +0000 UTC m=+1564.180918326"
Oct 30 22:03:42.565797 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:42.565698    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"
Oct 30 22:03:42.572046 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:42.571719    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"
Oct 30 22:03:42.825130 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:42.825043    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"
Oct 30 22:03:42.826358 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:42.826335    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"
Oct 30 22:03:43.018124 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:43.018080    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:43.230303 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:43.218745    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:44.838418 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:44.838365    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:45.064569 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:45.064526    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:46.616573 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:46.616450    4871 patch_prober.go:29] interesting pod/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.113:15021/healthz/ready\": read tcp 10.133.0.2:41082->10.133.0.113:15021: read: connection reset by peer" start-of-body=
Oct 30 22:03:46.616573 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:46.616518    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr" podUID="ef03dee1-e693-42a0-ace8-4a28e099b92b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.113:15021/healthz/ready\": read tcp 10.133.0.2:41082->10.133.0.113:15021: read: connection reset by peer"
Oct 30 22:03:46.656564 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:46.656523    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:46.960714 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:46.960681    4871 generic.go:356] "Generic (PLEG): container finished" podID="ef03dee1-e693-42a0-ace8-4a28e099b92b" containerID="2b49035c6e6dbaec7dd1491af63a36a98ff5b497551fe9664f34702e21c712c9" exitCode=0
Oct 30 22:03:46.960888 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:46.960813    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr" event={"ID":"ef03dee1-e693-42a0-ace8-4a28e099b92b","Type":"ContainerDied","Data":"2b49035c6e6dbaec7dd1491af63a36a98ff5b497551fe9664f34702e21c712c9"}
Oct 30 22:03:46.961226 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:46.961212    4871 scope.go:122] "RemoveContainer" containerID="2b49035c6e6dbaec7dd1491af63a36a98ff5b497551fe9664f34702e21c712c9"
Oct 30 22:03:46.962886 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:46.962860    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:03:46.962956 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:46.962925    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:03:46.962996 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:46.962962    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:03:47.970431 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:47.970353    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr" event={"ID":"ef03dee1-e693-42a0-ace8-4a28e099b92b","Type":"ContainerStarted","Data":"0dfe7f5862ac6b26a9e1bed74b43224d640b6f548b514e15e5eb1b2ad8e7746d"}
Oct 30 22:03:48.536829 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:48.536790    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"
Oct 30 22:03:48.537199 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:48.536963    4871 patch_prober.go:29] interesting pod/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.113:15021/healthz/ready\": dial tcp 10.133.0.113:15021: connect: connection refused" start-of-body=
Oct 30 22:03:48.537374 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:48.537273    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr" podUID="ef03dee1-e693-42a0-ace8-4a28e099b92b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.113:15021/healthz/ready\": dial tcp 10.133.0.113:15021: connect: connection refused"
Oct 30 22:03:49.536858 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:49.536821    4871 patch_prober.go:29] interesting pod/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.113:15021/healthz/ready\": dial tcp 10.133.0.113:15021: connect: connection refused" start-of-body=
Oct 30 22:03:49.537311 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:49.536891    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr" podUID="ef03dee1-e693-42a0-ace8-4a28e099b92b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.113:15021/healthz/ready\": dial tcp 10.133.0.113:15021: connect: connection refused"
Oct 30 22:03:50.379390 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:50.379338    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:50.539835 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:50.539799    4871 patch_prober.go:29] interesting pod/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.113:15021/healthz/ready\": dial tcp 10.133.0.113:15021: connect: connection refused" start-of-body=
Oct 30 22:03:50.540425 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:50.540391    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr" podUID="ef03dee1-e693-42a0-ace8-4a28e099b92b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.113:15021/healthz/ready\": dial tcp 10.133.0.113:15021: connect: connection refused"
Oct 30 22:03:51.271871 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:51.271839    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:51.536119 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:51.536033    4871 patch_prober.go:29] interesting pod/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.113:15021/healthz/ready\": dial tcp 10.133.0.113:15021: connect: connection refused" start-of-body=
Oct 30 22:03:51.536417 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:51.536386    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr" podUID="ef03dee1-e693-42a0-ace8-4a28e099b92b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.113:15021/healthz/ready\": dial tcp 10.133.0.113:15021: connect: connection refused"
Oct 30 22:03:51.734951 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:51.734908    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:51.862831 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:51.862790    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"]
Oct 30 22:03:51.867732 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:51.867709    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"
Oct 30 22:03:51.919246 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:51.919188    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-82b62e-openshift-default-dockercfg-fcbqx\""
Oct 30 22:03:52.027284 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:51.996158    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:52.027284 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:52.014700    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-istio-token\") pod \"e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4\" (UID: \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\") " pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"
Oct 30 22:03:52.027284 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:52.014745    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-istio-data\") pod \"e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4\" (UID: \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\") " pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"
Oct 30 22:03:52.027284 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:52.014776    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jczbn\" (UniqueName: \"kubernetes.io/projected/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-kube-api-access-jczbn\") pod \"e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4\" (UID: \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\") " pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"
Oct 30 22:03:52.027284 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:52.014878    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-credential-socket\") pod \"e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4\" (UID: \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\") " pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"
Oct 30 22:03:52.027284 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:52.014946    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-istio-envoy\") pod \"e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4\" (UID: \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\") " pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"
Oct 30 22:03:52.027284 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:52.014976    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-istio-podinfo\") pod \"e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4\" (UID: \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\") " pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"
Oct 30 22:03:52.027284 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:52.015121    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-workload-socket\") pod \"e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4\" (UID: \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\") " pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"
Oct 30 22:03:52.027284 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:52.015174    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-istiod-ca-cert\") pod \"e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4\" (UID: \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\") " pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"
Oct 30 22:03:52.027284 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:52.015203    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-workload-certs\") pod \"e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4\" (UID: \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\") " pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"
Oct 30 22:03:52.259749 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:52.259604    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:52.273633 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:52.273571    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-credential-socket\") pod \"e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4\" (UID: \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\") " pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"
Oct 30 22:03:52.273823 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:52.273684    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-istio-envoy\") pod \"e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4\" (UID: \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\") " pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"
Oct 30 22:03:52.273823 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:52.273721    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-istio-podinfo\") pod \"e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4\" (UID: \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\") " pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"
Oct 30 22:03:52.273944 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:52.273877    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-workload-socket\") pod \"e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4\" (UID: \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\") " pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"
Oct 30 22:03:52.273944 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:52.273929    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-istiod-ca-cert\") pod \"e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4\" (UID: \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\") " pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"
Oct 30 22:03:52.274076 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:52.273956    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-workload-certs\") pod \"e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4\" (UID: \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\") " pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"
Oct 30 22:03:52.274154 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:52.274130    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-istio-token\") pod \"e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4\" (UID: \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\") " pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"
Oct 30 22:03:52.274212 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:52.274161    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-istio-data\") pod \"e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4\" (UID: \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\") " pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"
Oct 30 22:03:52.274212 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:52.274187    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jczbn\" (UniqueName: \"kubernetes.io/projected/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-kube-api-access-jczbn\") pod \"e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4\" (UID: \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\") " pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"
Oct 30 22:03:52.275207 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:52.275181    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-credential-socket\") pod \"e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4\" (UID: \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\") " pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"
Oct 30 22:03:52.384995 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:52.384955    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-istio-envoy\") pod \"e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4\" (UID: \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\") " pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"
Oct 30 22:03:52.427804 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:52.398530    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-istiod-ca-cert\") pod \"e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4\" (UID: \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\") " pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"
Oct 30 22:03:52.427804 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:52.398841    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-workload-certs\") pod \"e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4\" (UID: \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\") " pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"
Oct 30 22:03:52.427804 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:52.420990    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-istio-data\") pod \"e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4\" (UID: \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\") " pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"
Oct 30 22:03:52.428510 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:52.428453    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-workload-socket\") pod \"e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4\" (UID: \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\") " pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"
Oct 30 22:03:52.628928 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:52.590468    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-istio-podinfo\") pod \"e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4\" (UID: \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\") " pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"
Oct 30 22:03:52.628928 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:52.594400    4871 patch_prober.go:29] interesting pod/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.113:15021/healthz/ready\": dial tcp 10.133.0.113:15021: connect: connection refused" start-of-body=
Oct 30 22:03:52.628928 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:52.628429    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr" podUID="ef03dee1-e693-42a0-ace8-4a28e099b92b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.113:15021/healthz/ready\": dial tcp 10.133.0.113:15021: connect: connection refused"
Oct 30 22:03:52.664262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:52.649834    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-istio-token\") pod \"e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4\" (UID: \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\") " pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"
Oct 30 22:03:52.665298 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:52.665208    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"]
Oct 30 22:03:52.693030 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:52.692998    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jczbn\" (UniqueName: \"kubernetes.io/projected/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-kube-api-access-jczbn\") pod \"e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4\" (UID: \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\") " pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"
Oct 30 22:03:52.785865 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:52.785824    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"
Oct 30 22:03:53.256761 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:53.256728    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"]
Oct 30 22:03:53.257145 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:03:53.257099    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod722963e5_d33d_42a6_9f76_7c9b8d2b2fe0.slice/crio-87a102e4b8e5a6a5e5291600a5dc786ab35b050e9ed0bda4e289ded1aaed2ac4 WatchSource:0}: Error finding container 87a102e4b8e5a6a5e5291600a5dc786ab35b050e9ed0bda4e289ded1aaed2ac4: Status 404 returned error can't find the container with id 87a102e4b8e5a6a5e5291600a5dc786ab35b050e9ed0bda4e289ded1aaed2ac4
Oct 30 22:03:53.260183 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:53.260149    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:03:53.260250 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:53.260218    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:03:53.260297 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:53.260273    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:03:53.536535 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:53.536412    4871 patch_prober.go:29] interesting pod/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.113:15021/healthz/ready\": dial tcp 10.133.0.113:15021: connect: connection refused" start-of-body=
Oct 30 22:03:53.536535 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:53.536471    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr" podUID="ef03dee1-e693-42a0-ace8-4a28e099b92b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.113:15021/healthz/ready\": dial tcp 10.133.0.113:15021: connect: connection refused"
Oct 30 22:03:54.194147 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:54.194107    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:54.206759 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:54.206720    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4" event={"ID":"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0","Type":"ContainerStarted","Data":"96f88fa306b5954d40cf5ef622ef33aadbe249cda9dbd69cb5452f869563ba45"}
Oct 30 22:03:54.206946 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:54.206782    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4" event={"ID":"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0","Type":"ContainerStarted","Data":"87a102e4b8e5a6a5e5291600a5dc786ab35b050e9ed0bda4e289ded1aaed2ac4"}
Oct 30 22:03:54.397259 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:54.394968    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:54.420711 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:54.416231    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4" podStartSLOduration=3.416205903 podStartE2EDuration="3.416205903s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:03:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:03:54.395332482 +0000 UTC m=+1576.572821104" watchObservedRunningTime="2026-10-30 22:03:54.416205903 +0000 UTC m=+1576.593694525"
Oct 30 22:03:54.600518 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:54.600357    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"
Oct 30 22:03:54.601343 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:54.601224    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"
Oct 30 22:03:54.602071 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:54.602048    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"
Oct 30 22:03:54.798262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:54.797482    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"
Oct 30 22:03:54.900262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:54.899433    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"
Oct 30 22:03:55.217698 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:55.217450    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"
Oct 30 22:03:55.218691 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:55.218660    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"
Oct 30 22:03:57.440260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:57.436812    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:57.681515 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:03:57.681445    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:01.732323 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:01.732281    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:02.009232 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:02.009138    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:02.961514 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:02.961471    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:03.061704 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:03.061666    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3bdc7244-nsmqj"]
Oct 30 22:04:03.169749 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:03.166781    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:03.179749 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:03.173896    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3bdc7244-nsmqj"]
Oct 30 22:04:03.962444 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:03.962414    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cc269490-09f2-4b2b-ba6d-df674223d731" path="/var/lib/kubelet/pods/cc269490-09f2-4b2b-ba6d-df674223d731/volumes"
Oct 30 22:04:05.221122 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:05.221082    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-5tn88"]
Oct 30 22:04:05.328913 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:05.328876    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4c260505-65bb6f8647-p5g2x"]
Oct 30 22:04:05.583646 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:04:05.583559    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:04:05.594214 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:05.594183    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:05.818856 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:05.788426    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:08.138646 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:08.138610    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:10.329675 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:10.329541    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-4c260505-65bb6f8647-p5g2x" podUID="d1717354-d285-4b73-a85e-8e8af064463b" containerName="payload-pre-processing" containerID="cri-o://8021e9fd80d19ec1f0e6c34b6eb348cc6bd78c7ba8cfbbcad57297803d83fdb0" gracePeriod=25
Oct 30 22:04:10.493068 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:10.493023    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-5tn88" podUID="4316833e-6973-4742-9eda-fb87f445a73b" containerName="payload-processing" containerID="cri-o://dfec834e8e05069bd20d95ea2565f3930e1af456049ed8c91ddf047e278ed86c" gracePeriod=25
Oct 30 22:04:12.859415 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:12.859371    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:14.588976 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:14.588931    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-4c260505-65bb6f8647-p5g2x" podUID="d1717354-d285-4b73-a85e-8e8af064463b" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:04:14.589432 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:14.588940    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-5tn88" podUID="4316833e-6973-4742-9eda-fb87f445a73b" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:04:17.515633 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:17.515591    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:22.279265 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:22.270522    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:24.589342 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:24.589302    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-5tn88" podUID="4316833e-6973-4742-9eda-fb87f445a73b" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:04:24.589777 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:24.589310    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-4c260505-65bb6f8647-p5g2x" podUID="d1717354-d285-4b73-a85e-8e8af064463b" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:04:26.759496 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:26.759455    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5"]
Oct 30 22:04:26.766005 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:26.765978    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5"
Oct 30 22:04:26.834231 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:26.834187    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5"]
Oct 30 22:04:26.835574 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:26.835543    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/904ace00-01d5-4e6b-ba97-d9e887b5da40-metrics-certs\") pod \"maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5\" (UID: \"904ace00-01d5-4e6b-ba97-d9e887b5da40\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5"
Oct 30 22:04:26.835971 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:26.835945    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/904ace00-01d5-4e6b-ba97-d9e887b5da40-openshift-service-ca\") pod \"maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5\" (UID: \"904ace00-01d5-4e6b-ba97-d9e887b5da40\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5"
Oct 30 22:04:26.836300 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:26.836268    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/904ace00-01d5-4e6b-ba97-d9e887b5da40-maas-api-tls\") pod \"maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5\" (UID: \"904ace00-01d5-4e6b-ba97-d9e887b5da40\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5"
Oct 30 22:04:26.836422 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:26.836357    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5xgk\" (UniqueName: \"kubernetes.io/projected/904ace00-01d5-4e6b-ba97-d9e887b5da40-kube-api-access-d5xgk\") pod \"maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5\" (UID: \"904ace00-01d5-4e6b-ba97-d9e887b5da40\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5"
Oct 30 22:04:26.872675 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:26.872636    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-a5f7b9-serving-cert\""
Oct 30 22:04:26.938185 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:26.938140    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/904ace00-01d5-4e6b-ba97-d9e887b5da40-openshift-service-ca\") pod \"maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5\" (UID: \"904ace00-01d5-4e6b-ba97-d9e887b5da40\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5"
Oct 30 22:04:26.939353 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:26.939221    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/904ace00-01d5-4e6b-ba97-d9e887b5da40-openshift-service-ca\") pod \"maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5\" (UID: \"904ace00-01d5-4e6b-ba97-d9e887b5da40\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5"
Oct 30 22:04:26.939693 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:26.939658    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/904ace00-01d5-4e6b-ba97-d9e887b5da40-maas-api-tls\") pod \"maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5\" (UID: \"904ace00-01d5-4e6b-ba97-d9e887b5da40\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5"
Oct 30 22:04:26.940535 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:26.939768    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5xgk\" (UniqueName: \"kubernetes.io/projected/904ace00-01d5-4e6b-ba97-d9e887b5da40-kube-api-access-d5xgk\") pod \"maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5\" (UID: \"904ace00-01d5-4e6b-ba97-d9e887b5da40\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5"
Oct 30 22:04:26.940535 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:26.939934    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/904ace00-01d5-4e6b-ba97-d9e887b5da40-metrics-certs\") pod \"maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5\" (UID: \"904ace00-01d5-4e6b-ba97-d9e887b5da40\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5"
Oct 30 22:04:26.940806 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:04:26.940782    4871 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-a5f7b9-serving-cert: secret "maas-api-e2e-api-b-a5f7b9-serving-cert" not found
Oct 30 22:04:26.940895 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:04:26.940870    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/904ace00-01d5-4e6b-ba97-d9e887b5da40-maas-api-tls podName:904ace00-01d5-4e6b-ba97-d9e887b5da40 nodeName:}" failed. No retries permitted until 2026-10-30 22:04:27.440849028 +0000 UTC m=+1609.618337644 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/904ace00-01d5-4e6b-ba97-d9e887b5da40-maas-api-tls") pod "maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5" (UID: "904ace00-01d5-4e6b-ba97-d9e887b5da40") : secret "maas-api-e2e-api-b-a5f7b9-serving-cert" not found
Oct 30 22:04:26.946217 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:26.945806    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/904ace00-01d5-4e6b-ba97-d9e887b5da40-metrics-certs\") pod \"maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5\" (UID: \"904ace00-01d5-4e6b-ba97-d9e887b5da40\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5"
Oct 30 22:04:26.969419 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:26.969388    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:26.999232 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:26.999195    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5xgk\" (UniqueName: \"kubernetes.io/projected/904ace00-01d5-4e6b-ba97-d9e887b5da40-kube-api-access-d5xgk\") pod \"maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5\" (UID: \"904ace00-01d5-4e6b-ba97-d9e887b5da40\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5"
Oct 30 22:04:27.173288 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:27.171583    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:27.454149 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:27.454065    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/904ace00-01d5-4e6b-ba97-d9e887b5da40-maas-api-tls\") pod \"maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5\" (UID: \"904ace00-01d5-4e6b-ba97-d9e887b5da40\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5"
Oct 30 22:04:27.456865 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:27.456836    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/904ace00-01d5-4e6b-ba97-d9e887b5da40-maas-api-tls\") pod \"maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5\" (UID: \"904ace00-01d5-4e6b-ba97-d9e887b5da40\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5"
Oct 30 22:04:27.682187 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:27.682153    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5"
Oct 30 22:04:28.135095 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:28.134987    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5"]
Oct 30 22:04:28.138667 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:04:28.138631    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod904ace00_01d5_4e6b_ba97_d9e887b5da40.slice/crio-c174006530408c291104ccb376da07b6c60bbc0c7eba254e37f33a90184dd128 WatchSource:0}: Error finding container c174006530408c291104ccb376da07b6c60bbc0c7eba254e37f33a90184dd128: Status 404 returned error can't find the container with id c174006530408c291104ccb376da07b6c60bbc0c7eba254e37f33a90184dd128
Oct 30 22:04:28.756056 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:28.756014    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5" event={"ID":"904ace00-01d5-4e6b-ba97-d9e887b5da40","Type":"ContainerStarted","Data":"c174006530408c291104ccb376da07b6c60bbc0c7eba254e37f33a90184dd128"}
Oct 30 22:04:29.788264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:29.788159    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5" event={"ID":"904ace00-01d5-4e6b-ba97-d9e887b5da40","Type":"ContainerStarted","Data":"59ed94f8f793d33a440af240a0a4add3d10510e0726aebf056b32e7dabb00ea7"}
Oct 30 22:04:29.790069 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:29.790018    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5"
Oct 30 22:04:29.916777 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:29.916587    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5" podStartSLOduration=2.894303431 podStartE2EDuration="3.916565913s" totalImagesPullingTime="1.022262482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:04:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:04:29.901750122 +0000 UTC m=+1612.079238739" watchObservedRunningTime="2026-10-30 22:04:29.916565913 +0000 UTC m=+1612.094054532"
Oct 30 22:04:30.634802 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:30.634771    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-4c260505-65bb6f8647-p5g2x"
Oct 30 22:04:30.756199 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:30.756166    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d1717354-d285-4b73-a85e-8e8af064463b-kube-api-access-mspfh\" (UniqueName: \"kubernetes.io/projected/d1717354-d285-4b73-a85e-8e8af064463b-kube-api-access-mspfh\") pod \"d1717354-d285-4b73-a85e-8e8af064463b\" (UID: \"d1717354-d285-4b73-a85e-8e8af064463b\") "
Oct 30 22:04:30.756436 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:30.756416    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d1717354-d285-4b73-a85e-8e8af064463b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1717354-d285-4b73-a85e-8e8af064463b-plugins-config-volume\") pod \"d1717354-d285-4b73-a85e-8e8af064463b\" (UID: \"d1717354-d285-4b73-a85e-8e8af064463b\") "
Oct 30 22:04:30.756965 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:30.756919    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1717354-d285-4b73-a85e-8e8af064463b-plugins-config-volume" pod "d1717354-d285-4b73-a85e-8e8af064463b" (UID: "d1717354-d285-4b73-a85e-8e8af064463b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:04:30.759875 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:30.759848    4871 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1717354-d285-4b73-a85e-8e8af064463b-plugins-config-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:04:30.772077 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:30.772021    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1717354-d285-4b73-a85e-8e8af064463b-kube-api-access-mspfh" pod "d1717354-d285-4b73-a85e-8e8af064463b" (UID: "d1717354-d285-4b73-a85e-8e8af064463b"). InnerVolumeSpecName "kube-api-access-mspfh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:04:30.799085 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:30.799000    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-5tn88"
Oct 30 22:04:30.805211 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:30.805181    4871 generic.go:356] "Generic (PLEG): container finished" podID="d1717354-d285-4b73-a85e-8e8af064463b" containerID="8021e9fd80d19ec1f0e6c34b6eb348cc6bd78c7ba8cfbbcad57297803d83fdb0" exitCode=0
Oct 30 22:04:30.805407 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:30.805254    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-4c260505-65bb6f8647-p5g2x"
Oct 30 22:04:30.805407 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:30.805269    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-4c260505-65bb6f8647-p5g2x" event={"ID":"d1717354-d285-4b73-a85e-8e8af064463b","Type":"ContainerDied","Data":"8021e9fd80d19ec1f0e6c34b6eb348cc6bd78c7ba8cfbbcad57297803d83fdb0"}
Oct 30 22:04:30.805407 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:30.805342    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-4c260505-65bb6f8647-p5g2x" event={"ID":"d1717354-d285-4b73-a85e-8e8af064463b","Type":"ContainerDied","Data":"4649c124ae7c7d8a521b7e45e04adf02793a4e711a12c5d9b09fa025db67f79f"}
Oct 30 22:04:30.805407 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:30.805377    4871 scope.go:122] "RemoveContainer" containerID="8021e9fd80d19ec1f0e6c34b6eb348cc6bd78c7ba8cfbbcad57297803d83fdb0"
Oct 30 22:04:30.806874 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:30.806850    4871 generic.go:356] "Generic (PLEG): container finished" podID="4316833e-6973-4742-9eda-fb87f445a73b" containerID="dfec834e8e05069bd20d95ea2565f3930e1af456049ed8c91ddf047e278ed86c" exitCode=0
Oct 30 22:04:30.806981 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:30.806886    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-5tn88"
Oct 30 22:04:30.807086 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:30.807061    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-5tn88" event={"ID":"4316833e-6973-4742-9eda-fb87f445a73b","Type":"ContainerDied","Data":"dfec834e8e05069bd20d95ea2565f3930e1af456049ed8c91ddf047e278ed86c"}
Oct 30 22:04:30.807142 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:30.807103    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-5tn88" event={"ID":"4316833e-6973-4742-9eda-fb87f445a73b","Type":"ContainerDied","Data":"6ddb3579a370156e8fd9b6fdfe2fc8bff8426605ae4dd8c8c4060cf21f6088f3"}
Oct 30 22:04:30.818970 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:30.818949    4871 scope.go:122] "RemoveContainer" containerID="8021e9fd80d19ec1f0e6c34b6eb348cc6bd78c7ba8cfbbcad57297803d83fdb0"
Oct 30 22:04:30.819324 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:04:30.819303    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8021e9fd80d19ec1f0e6c34b6eb348cc6bd78c7ba8cfbbcad57297803d83fdb0\": container with ID starting with 8021e9fd80d19ec1f0e6c34b6eb348cc6bd78c7ba8cfbbcad57297803d83fdb0 not found: ID does not exist" containerID="8021e9fd80d19ec1f0e6c34b6eb348cc6bd78c7ba8cfbbcad57297803d83fdb0"
Oct 30 22:04:30.819424 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:30.819332    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8021e9fd80d19ec1f0e6c34b6eb348cc6bd78c7ba8cfbbcad57297803d83fdb0"} err="failed to get container status \"8021e9fd80d19ec1f0e6c34b6eb348cc6bd78c7ba8cfbbcad57297803d83fdb0\": rpc error: code = NotFound desc = could not find container \"8021e9fd80d19ec1f0e6c34b6eb348cc6bd78c7ba8cfbbcad57297803d83fdb0\": container with ID starting with 8021e9fd80d19ec1f0e6c34b6eb348cc6bd78c7ba8cfbbcad57297803d83fdb0 not found: ID does not exist"
Oct 30 22:04:30.819424 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:30.819349    4871 scope.go:122] "RemoveContainer" containerID="dfec834e8e05069bd20d95ea2565f3930e1af456049ed8c91ddf047e278ed86c"
Oct 30 22:04:30.830209 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:30.830187    4871 scope.go:122] "RemoveContainer" containerID="dfec834e8e05069bd20d95ea2565f3930e1af456049ed8c91ddf047e278ed86c"
Oct 30 22:04:30.830609 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:04:30.830559    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfec834e8e05069bd20d95ea2565f3930e1af456049ed8c91ddf047e278ed86c\": container with ID starting with dfec834e8e05069bd20d95ea2565f3930e1af456049ed8c91ddf047e278ed86c not found: ID does not exist" containerID="dfec834e8e05069bd20d95ea2565f3930e1af456049ed8c91ddf047e278ed86c"
Oct 30 22:04:30.830609 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:30.830586    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfec834e8e05069bd20d95ea2565f3930e1af456049ed8c91ddf047e278ed86c"} err="failed to get container status \"dfec834e8e05069bd20d95ea2565f3930e1af456049ed8c91ddf047e278ed86c\": rpc error: code = NotFound desc = could not find container \"dfec834e8e05069bd20d95ea2565f3930e1af456049ed8c91ddf047e278ed86c\": container with ID starting with dfec834e8e05069bd20d95ea2565f3930e1af456049ed8c91ddf047e278ed86c not found: ID does not exist"
Oct 30 22:04:30.861590 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:30.861559    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mspfh\" (UniqueName: \"kubernetes.io/projected/d1717354-d285-4b73-a85e-8e8af064463b-kube-api-access-mspfh\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:04:30.969206 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:30.968500    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4316833e-6973-4742-9eda-fb87f445a73b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4316833e-6973-4742-9eda-fb87f445a73b-plugins-config-volume\") pod \"4316833e-6973-4742-9eda-fb87f445a73b\" (UID: \"4316833e-6973-4742-9eda-fb87f445a73b\") "
Oct 30 22:04:30.969206 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:30.968669    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4316833e-6973-4742-9eda-fb87f445a73b-kube-api-access-4mjhq\" (UniqueName: \"kubernetes.io/projected/4316833e-6973-4742-9eda-fb87f445a73b-kube-api-access-4mjhq\") pod \"4316833e-6973-4742-9eda-fb87f445a73b\" (UID: \"4316833e-6973-4742-9eda-fb87f445a73b\") "
Oct 30 22:04:30.973378 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:30.972616    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4316833e-6973-4742-9eda-fb87f445a73b-plugins-config-volume" pod "4316833e-6973-4742-9eda-fb87f445a73b" (UID: "4316833e-6973-4742-9eda-fb87f445a73b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:04:30.973603 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:30.973579    4871 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4316833e-6973-4742-9eda-fb87f445a73b-plugins-config-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:04:30.978597 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:30.977828    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4316833e-6973-4742-9eda-fb87f445a73b-kube-api-access-4mjhq" pod "4316833e-6973-4742-9eda-fb87f445a73b" (UID: "4316833e-6973-4742-9eda-fb87f445a73b"). InnerVolumeSpecName "kube-api-access-4mjhq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:04:31.039537 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:31.039483    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4c260505-65bb6f8647-p5g2x"]
Oct 30 22:04:31.044174 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:31.044135    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4c260505-65bb6f8647-p5g2x"]
Oct 30 22:04:31.075172 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:31.075131    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4mjhq\" (UniqueName: \"kubernetes.io/projected/4316833e-6973-4742-9eda-fb87f445a73b-kube-api-access-4mjhq\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:04:31.234445 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:31.234365    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-5tn88"]
Oct 30 22:04:31.244276 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:31.243259    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-5tn88"]
Oct 30 22:04:31.913124 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:31.913086    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:31.996347 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:31.991642    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4316833e-6973-4742-9eda-fb87f445a73b" path="/var/lib/kubelet/pods/4316833e-6973-4742-9eda-fb87f445a73b/volumes"
Oct 30 22:04:31.996347 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:31.992226    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d1717354-d285-4b73-a85e-8e8af064463b" path="/var/lib/kubelet/pods/d1717354-d285-4b73-a85e-8e8af064463b/volumes"
Oct 30 22:04:33.838048 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:33.838015    4871 generic.go:356] "Generic (PLEG): container finished" podID="f0cbb7ad-f150-4562-bb4c-ad9dce4545bd" containerID="d0b4495eb4008b7ed008ce1ded98730c0ab8dca8bdcb2dd342e6f7eed2fed0f3" exitCode=0
Oct 30 22:04:33.838505 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:33.838083    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm" event={"ID":"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd","Type":"ContainerDied","Data":"d0b4495eb4008b7ed008ce1ded98730c0ab8dca8bdcb2dd342e6f7eed2fed0f3"}
Oct 30 22:04:33.838589 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:33.838573    4871 scope.go:122] "RemoveContainer" containerID="d0b4495eb4008b7ed008ce1ded98730c0ab8dca8bdcb2dd342e6f7eed2fed0f3"
Oct 30 22:04:33.840210 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:33.840179    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:04:33.840359 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:33.840283    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:04:33.840359 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:33.840334    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:04:34.847775 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:34.847738    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm" event={"ID":"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd","Type":"ContainerStarted","Data":"2bfe9e580205f21e7c397d76d8dc80cb8774fbdb1a7f5ffc2b8690ae829b4fe4"}
Oct 30 22:04:35.169660 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:35.169629    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"
Oct 30 22:04:35.169843 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:35.169809    4871 patch_prober.go:29] interesting pod/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused" start-of-body=
Oct 30 22:04:35.169889 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:35.169858    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm" podUID="f0cbb7ad-f150-4562-bb4c-ad9dce4545bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused"
Oct 30 22:04:35.571781 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:04:35.571690    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:04:36.169199 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:36.169158    4871 patch_prober.go:29] interesting pod/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused" start-of-body=
Oct 30 22:04:36.169650 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:36.169223    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm" podUID="f0cbb7ad-f150-4562-bb4c-ad9dce4545bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused"
Oct 30 22:04:36.393461 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:36.393423    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5"]
Oct 30 22:04:36.394282 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:36.394255    4871 cpu_manager.go:414] "containerMap: removing container" podUID="4316833e-6973-4742-9eda-fb87f445a73b" containerName="payload-processing"
Oct 30 22:04:36.394282 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:36.394283    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4316833e-6973-4742-9eda-fb87f445a73b" containerName="payload-processing"
Oct 30 22:04:36.394485 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:36.394354    4871 cpu_manager.go:414] "containerMap: removing container" podUID="d1717354-d285-4b73-a85e-8e8af064463b" containerName="payload-pre-processing"
Oct 30 22:04:36.394485 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:36.394362    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d1717354-d285-4b73-a85e-8e8af064463b" containerName="payload-pre-processing"
Oct 30 22:04:36.394593 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:36.394556    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="d1717354-d285-4b73-a85e-8e8af064463b" containerName="payload-pre-processing"
Oct 30 22:04:36.394593 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:36.394579    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="4316833e-6973-4742-9eda-fb87f445a73b" containerName="payload-processing"
Oct 30 22:04:36.397573 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:36.397548    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5"
Oct 30 22:04:36.482004 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:36.481959    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd-maas-api-tls\") pod \"maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5\" (UID: \"0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5"
Oct 30 22:04:36.482210 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:36.482029    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd-metrics-certs\") pod \"maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5\" (UID: \"0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5"
Oct 30 22:04:36.482210 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:36.482171    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mljph\" (UniqueName: \"kubernetes.io/projected/0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd-kube-api-access-mljph\") pod \"maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5\" (UID: \"0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5"
Oct 30 22:04:36.482390 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:36.482361    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd-openshift-service-ca\") pod \"maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5\" (UID: \"0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5"
Oct 30 22:04:36.495589 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:36.495555    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-4cb675-serving-cert\""
Oct 30 22:04:36.500537 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:36.500502    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5"]
Oct 30 22:04:36.583634 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:36.583598    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd-maas-api-tls\") pod \"maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5\" (UID: \"0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5"
Oct 30 22:04:36.583840 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:36.583655    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd-metrics-certs\") pod \"maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5\" (UID: \"0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5"
Oct 30 22:04:36.583840 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:36.583700    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mljph\" (UniqueName: \"kubernetes.io/projected/0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd-kube-api-access-mljph\") pod \"maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5\" (UID: \"0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5"
Oct 30 22:04:36.583840 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:04:36.583779    4871 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-4cb675-serving-cert: secret "maas-api-e2e-api-a-4cb675-serving-cert" not found
Oct 30 22:04:36.583840 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:36.583799    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd-openshift-service-ca\") pod \"maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5\" (UID: \"0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5"
Oct 30 22:04:36.584066 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:04:36.583854    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd-maas-api-tls podName:0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd nodeName:}" failed. No retries permitted until 2026-10-30 22:04:37.083832074 +0000 UTC m=+1619.261320684 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd-maas-api-tls") pod "maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5" (UID: "0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd") : secret "maas-api-e2e-api-a-4cb675-serving-cert" not found
Oct 30 22:04:36.584571 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:36.584543    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd-openshift-service-ca\") pod \"maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5\" (UID: \"0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5"
Oct 30 22:04:36.586491 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:36.586465    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd-metrics-certs\") pod \"maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5\" (UID: \"0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5"
Oct 30 22:04:36.628404 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:36.626172    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:36.710308 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:36.709988    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mljph\" (UniqueName: \"kubernetes.io/projected/0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd-kube-api-access-mljph\") pod \"maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5\" (UID: \"0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5"
Oct 30 22:04:36.796226 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:36.796063    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:36.863232 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:36.863198    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5"
Oct 30 22:04:36.975315 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:36.972771    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:37.114261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:37.113802    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd-maas-api-tls\") pod \"maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5\" (UID: \"0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5"
Oct 30 22:04:37.169262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:37.168661    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd-maas-api-tls\") pod \"maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5\" (UID: \"0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5"
Oct 30 22:04:37.169262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:37.169195    4871 patch_prober.go:29] interesting pod/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused" start-of-body=
Oct 30 22:04:37.169806 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:37.169302    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm" podUID="f0cbb7ad-f150-4562-bb4c-ad9dce4545bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused"
Oct 30 22:04:37.338520 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:37.338436    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5"
Oct 30 22:04:37.491361 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:37.491323    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:37.748053 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:04:37.748019    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fce1e8b_09a8_4ba9_bc17_ba70a29f8ecd.slice/crio-2909a78aac91cf84b036ac584830e0b5c913501fa53c78fe461c448a424406ba WatchSource:0}: Error finding container 2909a78aac91cf84b036ac584830e0b5c913501fa53c78fe461c448a424406ba: Status 404 returned error can't find the container with id 2909a78aac91cf84b036ac584830e0b5c913501fa53c78fe461c448a424406ba
Oct 30 22:04:37.749136 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:37.749101    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5"]
Oct 30 22:04:37.937989 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:37.937953    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5" event={"ID":"0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd","Type":"ContainerStarted","Data":"2909a78aac91cf84b036ac584830e0b5c913501fa53c78fe461c448a424406ba"}
Oct 30 22:04:38.169332 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:38.169291    4871 patch_prober.go:29] interesting pod/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused" start-of-body=
Oct 30 22:04:38.169761 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:38.169361    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm" podUID="f0cbb7ad-f150-4562-bb4c-ad9dce4545bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused"
Oct 30 22:04:38.949505 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:38.949404    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5" event={"ID":"0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd","Type":"ContainerStarted","Data":"63357b25bf278c08446c7c3f842590dd72af36fd3349a44fedf3ed59f8c0f0e5"}
Oct 30 22:04:38.949692 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:38.949517    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5"
Oct 30 22:04:39.065824 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:39.065768    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5" podStartSLOduration=2.127485586 podStartE2EDuration="3.065752031s" totalImagesPullingTime="938.266445ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:04:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:04:39.06374938 +0000 UTC m=+1621.241237996" watchObservedRunningTime="2026-10-30 22:04:39.065752031 +0000 UTC m=+1621.243240648"
Oct 30 22:04:39.169385 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:39.169348    4871 patch_prober.go:29] interesting pod/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused" start-of-body=
Oct 30 22:04:39.169816 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:39.169407    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm" podUID="f0cbb7ad-f150-4562-bb4c-ad9dce4545bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused"
Oct 30 22:04:40.170150 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:40.170111    4871 patch_prober.go:29] interesting pod/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused" start-of-body=
Oct 30 22:04:40.170593 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:40.170199    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm" podUID="f0cbb7ad-f150-4562-bb4c-ad9dce4545bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused"
Oct 30 22:04:41.174024 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:41.173992    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"
Oct 30 22:04:41.174529 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:41.174415    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"
Oct 30 22:04:41.175098 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:41.175073    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"
Oct 30 22:04:41.638920 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:41.638883    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:44.966366 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:44.966335    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5"
Oct 30 22:04:46.547136 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:46.547094    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:51.317849 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:51.317806    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:51.566705 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:51.566555    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:56.180410 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:56.180376    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:58.046646 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:58.046606    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"]
Oct 30 22:04:58.047072 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:58.046865    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx" podUID="5ce15ea8-02f4-4b1a-8670-937679cd7e57" containerName="istio-proxy" containerID="cri-o://4a8517c00ea94add2fac3695fcad02e46e56ca62c0e957b7f6050c5a49b0e9ed" gracePeriod=30
Oct 30 22:04:58.092419 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:58.092380    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"]
Oct 30 22:04:58.092635 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:58.092589    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2" podUID="85541021-4964-4ce1-b471-9df326efc190" containerName="istio-proxy" containerID="cri-o://eb571be43aba8fc65f6de5098c2a28fa0a8caeaffaf7ca3252034053c1e7e1d1" gracePeriod=30
Oct 30 22:04:58.551086 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:58.551050    4871 patch_prober.go:29] interesting pod/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:04:58.551284 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:04:58.551122    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2" podUID="85541021-4964-4ce1-b471-9df326efc190" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:05:00.765775 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:00.765742    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:03.367773 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.367732    4871 generic.go:356] "Generic (PLEG): container finished" podID="5ce15ea8-02f4-4b1a-8670-937679cd7e57" containerID="4a8517c00ea94add2fac3695fcad02e46e56ca62c0e957b7f6050c5a49b0e9ed" exitCode=0
Oct 30 22:05:03.368332 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.368059    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx" event={"ID":"5ce15ea8-02f4-4b1a-8670-937679cd7e57","Type":"ContainerDied","Data":"4a8517c00ea94add2fac3695fcad02e46e56ca62c0e957b7f6050c5a49b0e9ed"}
Oct 30 22:05:03.371079 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.371051    4871 generic.go:356] "Generic (PLEG): container finished" podID="85541021-4964-4ce1-b471-9df326efc190" containerID="eb571be43aba8fc65f6de5098c2a28fa0a8caeaffaf7ca3252034053c1e7e1d1" exitCode=0
Oct 30 22:05:03.371200 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.371159    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2" event={"ID":"85541021-4964-4ce1-b471-9df326efc190","Type":"ContainerDied","Data":"eb571be43aba8fc65f6de5098c2a28fa0a8caeaffaf7ca3252034053c1e7e1d1"}
Oct 30 22:05:03.401011 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.400975    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"
Oct 30 22:05:03.437170 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.437144    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"
Oct 30 22:05:03.495140 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.495101    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5ce15ea8-02f4-4b1a-8670-937679cd7e57-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ce15ea8-02f4-4b1a-8670-937679cd7e57-istiod-ca-cert\") pod \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\" (UID: \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\") "
Oct 30 22:05:03.495140 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.495152    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85541021-4964-4ce1-b471-9df326efc190-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85541021-4964-4ce1-b471-9df326efc190-credential-socket\") pod \"85541021-4964-4ce1-b471-9df326efc190\" (UID: \"85541021-4964-4ce1-b471-9df326efc190\") "
Oct 30 22:05:03.495453 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.495211    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85541021-4964-4ce1-b471-9df326efc190-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85541021-4964-4ce1-b471-9df326efc190-istio-envoy\") pod \"85541021-4964-4ce1-b471-9df326efc190\" (UID: \"85541021-4964-4ce1-b471-9df326efc190\") "
Oct 30 22:05:03.495453 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.495273    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5ce15ea8-02f4-4b1a-8670-937679cd7e57-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ce15ea8-02f4-4b1a-8670-937679cd7e57-istio-podinfo\") pod \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\" (UID: \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\") "
Oct 30 22:05:03.495453 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.495314    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5ce15ea8-02f4-4b1a-8670-937679cd7e57-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ce15ea8-02f4-4b1a-8670-937679cd7e57-credential-socket\") pod \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\" (UID: \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\") "
Oct 30 22:05:03.495453 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.495347    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/85541021-4964-4ce1-b471-9df326efc190-istio-token\" (UniqueName: \"kubernetes.io/projected/85541021-4964-4ce1-b471-9df326efc190-istio-token\") pod \"85541021-4964-4ce1-b471-9df326efc190\" (UID: \"85541021-4964-4ce1-b471-9df326efc190\") "
Oct 30 22:05:03.495453 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.495390    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5ce15ea8-02f4-4b1a-8670-937679cd7e57-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ce15ea8-02f4-4b1a-8670-937679cd7e57-workload-certs\") pod \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\" (UID: \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\") "
Oct 30 22:05:03.495453 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.495428    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85541021-4964-4ce1-b471-9df326efc190-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85541021-4964-4ce1-b471-9df326efc190-workload-certs\") pod \"85541021-4964-4ce1-b471-9df326efc190\" (UID: \"85541021-4964-4ce1-b471-9df326efc190\") "
Oct 30 22:05:03.495759 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.495533    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85541021-4964-4ce1-b471-9df326efc190-credential-socket" pod "85541021-4964-4ce1-b471-9df326efc190" (UID: "85541021-4964-4ce1-b471-9df326efc190"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:03.495759 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.495545    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5ce15ea8-02f4-4b1a-8670-937679cd7e57-kube-api-access-h68z5\" (UniqueName: \"kubernetes.io/projected/5ce15ea8-02f4-4b1a-8670-937679cd7e57-kube-api-access-h68z5\") pod \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\" (UID: \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\") "
Oct 30 22:05:03.495759 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.495591    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ce15ea8-02f4-4b1a-8670-937679cd7e57-istiod-ca-cert" pod "5ce15ea8-02f4-4b1a-8670-937679cd7e57" (UID: "5ce15ea8-02f4-4b1a-8670-937679cd7e57"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:05:03.495759 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.495611    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/85541021-4964-4ce1-b471-9df326efc190-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85541021-4964-4ce1-b471-9df326efc190-istiod-ca-cert\") pod \"85541021-4964-4ce1-b471-9df326efc190\" (UID: \"85541021-4964-4ce1-b471-9df326efc190\") "
Oct 30 22:05:03.495759 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.495634    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/85541021-4964-4ce1-b471-9df326efc190-kube-api-access-tsxkc\" (UniqueName: \"kubernetes.io/projected/85541021-4964-4ce1-b471-9df326efc190-kube-api-access-tsxkc\") pod \"85541021-4964-4ce1-b471-9df326efc190\" (UID: \"85541021-4964-4ce1-b471-9df326efc190\") "
Oct 30 22:05:03.495759 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.495671    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/85541021-4964-4ce1-b471-9df326efc190-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85541021-4964-4ce1-b471-9df326efc190-istio-podinfo\") pod \"85541021-4964-4ce1-b471-9df326efc190\" (UID: \"85541021-4964-4ce1-b471-9df326efc190\") "
Oct 30 22:05:03.496068 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.495771    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5ce15ea8-02f4-4b1a-8670-937679cd7e57-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ce15ea8-02f4-4b1a-8670-937679cd7e57-workload-socket\") pod \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\" (UID: \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\") "
Oct 30 22:05:03.496068 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.495807    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ce15ea8-02f4-4b1a-8670-937679cd7e57-credential-socket" pod "5ce15ea8-02f4-4b1a-8670-937679cd7e57" (UID: "5ce15ea8-02f4-4b1a-8670-937679cd7e57"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:03.496068 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.495917    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5ce15ea8-02f4-4b1a-8670-937679cd7e57-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ce15ea8-02f4-4b1a-8670-937679cd7e57-istio-envoy\") pod \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\" (UID: \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\") "
Oct 30 22:05:03.496068 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.495929    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ce15ea8-02f4-4b1a-8670-937679cd7e57-workload-certs" pod "5ce15ea8-02f4-4b1a-8670-937679cd7e57" (UID: "5ce15ea8-02f4-4b1a-8670-937679cd7e57"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:03.496068 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.495954    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85541021-4964-4ce1-b471-9df326efc190-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85541021-4964-4ce1-b471-9df326efc190-workload-socket\") pod \"85541021-4964-4ce1-b471-9df326efc190\" (UID: \"85541021-4964-4ce1-b471-9df326efc190\") "
Oct 30 22:05:03.496068 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.495967    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85541021-4964-4ce1-b471-9df326efc190-workload-certs" pod "85541021-4964-4ce1-b471-9df326efc190" (UID: "85541021-4964-4ce1-b471-9df326efc190"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:03.496068 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.496010    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5ce15ea8-02f4-4b1a-8670-937679cd7e57-istio-token\" (UniqueName: \"kubernetes.io/projected/5ce15ea8-02f4-4b1a-8670-937679cd7e57-istio-token\") pod \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\" (UID: \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\") "
Oct 30 22:05:03.496450 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.496076    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5ce15ea8-02f4-4b1a-8670-937679cd7e57-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ce15ea8-02f4-4b1a-8670-937679cd7e57-istio-data\") pod \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\" (UID: \"5ce15ea8-02f4-4b1a-8670-937679cd7e57\") "
Oct 30 22:05:03.496450 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.496136    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85541021-4964-4ce1-b471-9df326efc190-workload-socket" pod "85541021-4964-4ce1-b471-9df326efc190" (UID: "85541021-4964-4ce1-b471-9df326efc190"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:03.496450 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.496208    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ce15ea8-02f4-4b1a-8670-937679cd7e57-workload-socket" pod "5ce15ea8-02f4-4b1a-8670-937679cd7e57" (UID: "5ce15ea8-02f4-4b1a-8670-937679cd7e57"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:03.496450 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.496260    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85541021-4964-4ce1-b471-9df326efc190-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85541021-4964-4ce1-b471-9df326efc190-istio-data\") pod \"85541021-4964-4ce1-b471-9df326efc190\" (UID: \"85541021-4964-4ce1-b471-9df326efc190\") "
Oct 30 22:05:03.496674 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.496535    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ce15ea8-02f4-4b1a-8670-937679cd7e57-istio-data" pod "5ce15ea8-02f4-4b1a-8670-937679cd7e57" (UID: "5ce15ea8-02f4-4b1a-8670-937679cd7e57"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:03.496792 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.496757    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85541021-4964-4ce1-b471-9df326efc190-istio-data" pod "85541021-4964-4ce1-b471-9df326efc190" (UID: "85541021-4964-4ce1-b471-9df326efc190"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:03.497139 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.497110    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85541021-4964-4ce1-b471-9df326efc190-istiod-ca-cert" pod "85541021-4964-4ce1-b471-9df326efc190" (UID: "85541021-4964-4ce1-b471-9df326efc190"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:05:03.497281 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.497187    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85541021-4964-4ce1-b471-9df326efc190-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:03.497281 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.497209    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ce15ea8-02f4-4b1a-8670-937679cd7e57-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:03.497281 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.497224    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85541021-4964-4ce1-b471-9df326efc190-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:03.497281 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.497252    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ce15ea8-02f4-4b1a-8670-937679cd7e57-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:03.497281 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.497269    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85541021-4964-4ce1-b471-9df326efc190-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:03.497578 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.497284    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ce15ea8-02f4-4b1a-8670-937679cd7e57-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:03.497578 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.497298    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ce15ea8-02f4-4b1a-8670-937679cd7e57-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:03.497578 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.497310    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85541021-4964-4ce1-b471-9df326efc190-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:03.497578 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.497324    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ce15ea8-02f4-4b1a-8670-937679cd7e57-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:03.498512 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.498483    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ce15ea8-02f4-4b1a-8670-937679cd7e57-kube-api-access-h68z5" pod "5ce15ea8-02f4-4b1a-8670-937679cd7e57" (UID: "5ce15ea8-02f4-4b1a-8670-937679cd7e57"). InnerVolumeSpecName "kube-api-access-h68z5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:05:03.498630 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.498581    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5ce15ea8-02f4-4b1a-8670-937679cd7e57-istio-podinfo" pod "5ce15ea8-02f4-4b1a-8670-937679cd7e57" (UID: "5ce15ea8-02f4-4b1a-8670-937679cd7e57"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:05:03.498708 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.498666    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ce15ea8-02f4-4b1a-8670-937679cd7e57-istio-envoy" pod "5ce15ea8-02f4-4b1a-8670-937679cd7e57" (UID: "5ce15ea8-02f4-4b1a-8670-937679cd7e57"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:03.498776 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.498697    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/85541021-4964-4ce1-b471-9df326efc190-istio-podinfo" pod "85541021-4964-4ce1-b471-9df326efc190" (UID: "85541021-4964-4ce1-b471-9df326efc190"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:05:03.499009 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.498989    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85541021-4964-4ce1-b471-9df326efc190-istio-envoy" pod "85541021-4964-4ce1-b471-9df326efc190" (UID: "85541021-4964-4ce1-b471-9df326efc190"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:03.499596 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.499573    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85541021-4964-4ce1-b471-9df326efc190-istio-token" pod "85541021-4964-4ce1-b471-9df326efc190" (UID: "85541021-4964-4ce1-b471-9df326efc190"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:05:03.499685 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.499630    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85541021-4964-4ce1-b471-9df326efc190-kube-api-access-tsxkc" pod "85541021-4964-4ce1-b471-9df326efc190" (UID: "85541021-4964-4ce1-b471-9df326efc190"). InnerVolumeSpecName "kube-api-access-tsxkc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:05:03.499748 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.499732    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ce15ea8-02f4-4b1a-8670-937679cd7e57-istio-token" pod "5ce15ea8-02f4-4b1a-8670-937679cd7e57" (UID: "5ce15ea8-02f4-4b1a-8670-937679cd7e57"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:05:03.599156 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.599115    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85541021-4964-4ce1-b471-9df326efc190-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:03.599156 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.599160    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tsxkc\" (UniqueName: \"kubernetes.io/projected/85541021-4964-4ce1-b471-9df326efc190-kube-api-access-tsxkc\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:03.599399 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.599178    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85541021-4964-4ce1-b471-9df326efc190-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:03.599399 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.599196    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ce15ea8-02f4-4b1a-8670-937679cd7e57-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:03.599399 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.599211    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ce15ea8-02f4-4b1a-8670-937679cd7e57-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:03.599399 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.599224    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85541021-4964-4ce1-b471-9df326efc190-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:03.599399 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.599280    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ce15ea8-02f4-4b1a-8670-937679cd7e57-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:03.599399 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.599296    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85541021-4964-4ce1-b471-9df326efc190-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:03.599399 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:03.599310    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h68z5\" (UniqueName: \"kubernetes.io/projected/5ce15ea8-02f4-4b1a-8670-937679cd7e57-kube-api-access-h68z5\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:04.381566 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:04.381533    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"
Oct 30 22:05:04.381566 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:04.381551    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx" event={"ID":"5ce15ea8-02f4-4b1a-8670-937679cd7e57","Type":"ContainerDied","Data":"834e37586623ab61db68805d51e8a86df9eecc4f74ec739a2388de6a48bf4d6b"}
Oct 30 22:05:04.382030 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:04.381617    4871 scope.go:122] "RemoveContainer" containerID="4a8517c00ea94add2fac3695fcad02e46e56ca62c0e957b7f6050c5a49b0e9ed"
Oct 30 22:05:04.383412 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:04.383382    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2" event={"ID":"85541021-4964-4ce1-b471-9df326efc190","Type":"ContainerDied","Data":"ab97931147a1aa7bb2f9fc2d18f0812ca52b905b4ef9bbf9ab3219dbf08407c7"}
Oct 30 22:05:04.383536 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:04.383390    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"
Oct 30 22:05:04.392992 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:04.392971    4871 scope.go:122] "RemoveContainer" containerID="eb571be43aba8fc65f6de5098c2a28fa0a8caeaffaf7ca3252034053c1e7e1d1"
Oct 30 22:05:04.600248 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:04.600211    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"]
Oct 30 22:05:04.647573 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:04.647472    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-442wx"]
Oct 30 22:05:04.754349 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:04.754299    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"]
Oct 30 22:05:04.760217 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:04.760189    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-h8dt2"]
Oct 30 22:05:05.571070 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:05:05.571028    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:05:05.769259 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:05.762130    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:05.963526 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:05.963488    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5ce15ea8-02f4-4b1a-8670-937679cd7e57" path="/var/lib/kubelet/pods/5ce15ea8-02f4-4b1a-8670-937679cd7e57/volumes"
Oct 30 22:05:05.964059 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:05.964044    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="85541021-4964-4ce1-b471-9df326efc190" path="/var/lib/kubelet/pods/85541021-4964-4ce1-b471-9df326efc190/volumes"
Oct 30 22:05:08.357144 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:08.357105    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:09.081291 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:09.081259    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:09.730273 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:09.728892    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"]
Oct 30 22:05:09.730273 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:09.730012    4871 cpu_manager.go:414] "containerMap: removing container" podUID="85541021-4964-4ce1-b471-9df326efc190" containerName="istio-proxy"
Oct 30 22:05:09.730273 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:09.730035    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="85541021-4964-4ce1-b471-9df326efc190" containerName="istio-proxy"
Oct 30 22:05:09.730273 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:09.730115    4871 cpu_manager.go:414] "containerMap: removing container" podUID="5ce15ea8-02f4-4b1a-8670-937679cd7e57" containerName="istio-proxy"
Oct 30 22:05:09.730273 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:09.730124    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5ce15ea8-02f4-4b1a-8670-937679cd7e57" containerName="istio-proxy"
Oct 30 22:05:09.730999 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:09.730295    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="5ce15ea8-02f4-4b1a-8670-937679cd7e57" containerName="istio-proxy"
Oct 30 22:05:09.730999 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:09.730318    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="85541021-4964-4ce1-b471-9df326efc190" containerName="istio-proxy"
Oct 30 22:05:09.790262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:09.789411    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"
Oct 30 22:05:09.950263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:09.947087    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:10.011549 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:10.011446    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1743672-f86e-4bb4-a827-99344823cc30-workload-certs\") pod \"e2e-api-b-97f097-openshift-default-589b54574c-469zm\" (UID: \"d1743672-f86e-4bb4-a827-99344823cc30\") " pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"
Oct 30 22:05:10.011777 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:10.011752    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1743672-f86e-4bb4-a827-99344823cc30-istio-podinfo\") pod \"e2e-api-b-97f097-openshift-default-589b54574c-469zm\" (UID: \"d1743672-f86e-4bb4-a827-99344823cc30\") " pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"
Oct 30 22:05:10.011849 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:10.011797    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1743672-f86e-4bb4-a827-99344823cc30-istiod-ca-cert\") pod \"e2e-api-b-97f097-openshift-default-589b54574c-469zm\" (UID: \"d1743672-f86e-4bb4-a827-99344823cc30\") " pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"
Oct 30 22:05:10.011903 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:10.011890    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1743672-f86e-4bb4-a827-99344823cc30-workload-socket\") pod \"e2e-api-b-97f097-openshift-default-589b54574c-469zm\" (UID: \"d1743672-f86e-4bb4-a827-99344823cc30\") " pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"
Oct 30 22:05:10.012010 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:10.011990    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9f86p\" (UniqueName: \"kubernetes.io/projected/d1743672-f86e-4bb4-a827-99344823cc30-kube-api-access-9f86p\") pod \"e2e-api-b-97f097-openshift-default-589b54574c-469zm\" (UID: \"d1743672-f86e-4bb4-a827-99344823cc30\") " pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"
Oct 30 22:05:10.012074 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:10.012038    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1743672-f86e-4bb4-a827-99344823cc30-istio-token\") pod \"e2e-api-b-97f097-openshift-default-589b54574c-469zm\" (UID: \"d1743672-f86e-4bb4-a827-99344823cc30\") " pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"
Oct 30 22:05:10.012175 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:10.012156    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1743672-f86e-4bb4-a827-99344823cc30-istio-data\") pod \"e2e-api-b-97f097-openshift-default-589b54574c-469zm\" (UID: \"d1743672-f86e-4bb4-a827-99344823cc30\") " pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"
Oct 30 22:05:10.012256 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:10.012201    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1743672-f86e-4bb4-a827-99344823cc30-istio-envoy\") pod \"e2e-api-b-97f097-openshift-default-589b54574c-469zm\" (UID: \"d1743672-f86e-4bb4-a827-99344823cc30\") " pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"
Oct 30 22:05:10.012390 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:10.012371    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1743672-f86e-4bb4-a827-99344823cc30-credential-socket\") pod \"e2e-api-b-97f097-openshift-default-589b54574c-469zm\" (UID: \"d1743672-f86e-4bb4-a827-99344823cc30\") " pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"
Oct 30 22:05:10.012627 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:10.012601    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-97f097-openshift-default-dockercfg-vtfjn\""
Oct 30 22:05:10.178273 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:10.177499    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:10.349317 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:10.294561    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1743672-f86e-4bb4-a827-99344823cc30-istio-podinfo\") pod \"e2e-api-b-97f097-openshift-default-589b54574c-469zm\" (UID: \"d1743672-f86e-4bb4-a827-99344823cc30\") " pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"
Oct 30 22:05:10.398565 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:10.398519    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1743672-f86e-4bb4-a827-99344823cc30-istiod-ca-cert\") pod \"e2e-api-b-97f097-openshift-default-589b54574c-469zm\" (UID: \"d1743672-f86e-4bb4-a827-99344823cc30\") " pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"
Oct 30 22:05:10.398944 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:10.398919    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1743672-f86e-4bb4-a827-99344823cc30-workload-socket\") pod \"e2e-api-b-97f097-openshift-default-589b54574c-469zm\" (UID: \"d1743672-f86e-4bb4-a827-99344823cc30\") " pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"
Oct 30 22:05:10.399114 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:10.399100    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9f86p\" (UniqueName: \"kubernetes.io/projected/d1743672-f86e-4bb4-a827-99344823cc30-kube-api-access-9f86p\") pod \"e2e-api-b-97f097-openshift-default-589b54574c-469zm\" (UID: \"d1743672-f86e-4bb4-a827-99344823cc30\") " pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"
Oct 30 22:05:10.399226 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:10.399213    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1743672-f86e-4bb4-a827-99344823cc30-istio-token\") pod \"e2e-api-b-97f097-openshift-default-589b54574c-469zm\" (UID: \"d1743672-f86e-4bb4-a827-99344823cc30\") " pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"
Oct 30 22:05:10.399526 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:10.399502    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1743672-f86e-4bb4-a827-99344823cc30-istio-data\") pod \"e2e-api-b-97f097-openshift-default-589b54574c-469zm\" (UID: \"d1743672-f86e-4bb4-a827-99344823cc30\") " pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"
Oct 30 22:05:10.399689 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:10.399665    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1743672-f86e-4bb4-a827-99344823cc30-istio-envoy\") pod \"e2e-api-b-97f097-openshift-default-589b54574c-469zm\" (UID: \"d1743672-f86e-4bb4-a827-99344823cc30\") " pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"
Oct 30 22:05:10.399993 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:10.399970    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1743672-f86e-4bb4-a827-99344823cc30-credential-socket\") pod \"e2e-api-b-97f097-openshift-default-589b54574c-469zm\" (UID: \"d1743672-f86e-4bb4-a827-99344823cc30\") " pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"
Oct 30 22:05:10.477273 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:10.469451    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1743672-f86e-4bb4-a827-99344823cc30-workload-certs\") pod \"e2e-api-b-97f097-openshift-default-589b54574c-469zm\" (UID: \"d1743672-f86e-4bb4-a827-99344823cc30\") " pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"
Oct 30 22:05:10.477273 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:10.300816    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"]
Oct 30 22:05:10.477273 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:10.471265    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:10.495275 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:10.490250    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1743672-f86e-4bb4-a827-99344823cc30-istiod-ca-cert\") pod \"e2e-api-b-97f097-openshift-default-589b54574c-469zm\" (UID: \"d1743672-f86e-4bb4-a827-99344823cc30\") " pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"
Oct 30 22:05:10.504435 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:10.502826    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1743672-f86e-4bb4-a827-99344823cc30-workload-socket\") pod \"e2e-api-b-97f097-openshift-default-589b54574c-469zm\" (UID: \"d1743672-f86e-4bb4-a827-99344823cc30\") " pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"
Oct 30 22:05:10.510224 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:10.510192    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1743672-f86e-4bb4-a827-99344823cc30-istio-podinfo\") pod \"e2e-api-b-97f097-openshift-default-589b54574c-469zm\" (UID: \"d1743672-f86e-4bb4-a827-99344823cc30\") " pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"
Oct 30 22:05:10.513311 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:10.512739    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1743672-f86e-4bb4-a827-99344823cc30-istio-data\") pod \"e2e-api-b-97f097-openshift-default-589b54574c-469zm\" (UID: \"d1743672-f86e-4bb4-a827-99344823cc30\") " pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"
Oct 30 22:05:10.513311 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:10.513018    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1743672-f86e-4bb4-a827-99344823cc30-workload-certs\") pod \"e2e-api-b-97f097-openshift-default-589b54574c-469zm\" (UID: \"d1743672-f86e-4bb4-a827-99344823cc30\") " pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"
Oct 30 22:05:10.513311 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:10.513273    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1743672-f86e-4bb4-a827-99344823cc30-credential-socket\") pod \"e2e-api-b-97f097-openshift-default-589b54574c-469zm\" (UID: \"d1743672-f86e-4bb4-a827-99344823cc30\") " pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"
Oct 30 22:05:10.551960 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:10.544416    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9f86p\" (UniqueName: \"kubernetes.io/projected/d1743672-f86e-4bb4-a827-99344823cc30-kube-api-access-9f86p\") pod \"e2e-api-b-97f097-openshift-default-589b54574c-469zm\" (UID: \"d1743672-f86e-4bb4-a827-99344823cc30\") " pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"
Oct 30 22:05:10.551960 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:10.548996    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1743672-f86e-4bb4-a827-99344823cc30-istio-envoy\") pod \"e2e-api-b-97f097-openshift-default-589b54574c-469zm\" (UID: \"d1743672-f86e-4bb4-a827-99344823cc30\") " pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"
Oct 30 22:05:10.551960 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:10.551760    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1743672-f86e-4bb4-a827-99344823cc30-istio-token\") pod \"e2e-api-b-97f097-openshift-default-589b54574c-469zm\" (UID: \"d1743672-f86e-4bb4-a827-99344823cc30\") " pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"
Oct 30 22:05:10.845648 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:10.845597    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"
Oct 30 22:05:11.433060 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:05:11.433021    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1743672_f86e_4bb4_a827_99344823cc30.slice/crio-2dc766b24fa4e8b8189013aacda6afb8ef229099a8269b600d2573d400dc8c19 WatchSource:0}: Error finding container 2dc766b24fa4e8b8189013aacda6afb8ef229099a8269b600d2573d400dc8c19: Status 404 returned error can't find the container with id 2dc766b24fa4e8b8189013aacda6afb8ef229099a8269b600d2573d400dc8c19
Oct 30 22:05:11.453099 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:11.450762    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:05:11.453099 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:11.450865    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:05:11.453099 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:11.450911    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:05:11.477153 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:11.477112    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"]
Oct 30 22:05:11.571331 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:11.567568    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm" event={"ID":"d1743672-f86e-4bb4-a827-99344823cc30","Type":"ContainerStarted","Data":"2dc766b24fa4e8b8189013aacda6afb8ef229099a8269b600d2573d400dc8c19"}
Oct 30 22:05:12.331158 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:12.331122    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:12.579220 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:12.579181    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm" event={"ID":"d1743672-f86e-4bb4-a827-99344823cc30","Type":"ContainerStarted","Data":"4363fcea3e6f82ba8fc2ddac054384b0cc3df2821c66111ae176c14635aaea1c"}
Oct 30 22:05:12.699615 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:12.699556    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm" podStartSLOduration=3.6995407350000002 podStartE2EDuration="3.699540735s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:05:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:05:12.695410638 +0000 UTC m=+1654.872899255" watchObservedRunningTime="2026-10-30 22:05:12.699540735 +0000 UTC m=+1654.877029425"
Oct 30 22:05:12.856261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:12.846903    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"
Oct 30 22:05:12.861261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:12.858294    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:12.903264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:12.898481    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"
Oct 30 22:05:13.239777 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:13.239726    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:13.587138 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:13.587059    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"
Oct 30 22:05:13.588171 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:13.588149    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"
Oct 30 22:05:15.384498 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:15.384341    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"]
Oct 30 22:05:15.385003 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:15.384815    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql" podUID="33920f19-5306-4275-b275-3433be348592" containerName="istio-proxy" containerID="cri-o://903b49276da1b65dc699be10ae7773ed65cba1c379157ea01016890a10e3fb8b" gracePeriod=30
Oct 30 22:05:15.584962 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:15.584895    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:15.941227 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:15.941188    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:16.124102 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:16.124065    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:18.276313 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:18.271958    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:19.390411 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:19.390372    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:19.919323 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:19.919000    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:20.966074 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:20.965997    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"
Oct 30 22:05:20.970021 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:20.969987    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33920f19-5306-4275-b275-3433be348592-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33920f19-5306-4275-b275-3433be348592-credential-socket\") pod \"33920f19-5306-4275-b275-3433be348592\" (UID: \"33920f19-5306-4275-b275-3433be348592\") "
Oct 30 22:05:20.970158 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:20.970043    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33920f19-5306-4275-b275-3433be348592-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33920f19-5306-4275-b275-3433be348592-workload-socket\") pod \"33920f19-5306-4275-b275-3433be348592\" (UID: \"33920f19-5306-4275-b275-3433be348592\") "
Oct 30 22:05:20.970158 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:20.970094    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33920f19-5306-4275-b275-3433be348592-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33920f19-5306-4275-b275-3433be348592-workload-certs\") pod \"33920f19-5306-4275-b275-3433be348592\" (UID: \"33920f19-5306-4275-b275-3433be348592\") "
Oct 30 22:05:20.970158 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:20.970125    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/33920f19-5306-4275-b275-3433be348592-istio-token\" (UniqueName: \"kubernetes.io/projected/33920f19-5306-4275-b275-3433be348592-istio-token\") pod \"33920f19-5306-4275-b275-3433be348592\" (UID: \"33920f19-5306-4275-b275-3433be348592\") "
Oct 30 22:05:20.970289 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:20.970193    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33920f19-5306-4275-b275-3433be348592-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33920f19-5306-4275-b275-3433be348592-istio-envoy\") pod \"33920f19-5306-4275-b275-3433be348592\" (UID: \"33920f19-5306-4275-b275-3433be348592\") "
Oct 30 22:05:20.970289 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:20.970223    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/33920f19-5306-4275-b275-3433be348592-kube-api-access-ssq4g\" (UniqueName: \"kubernetes.io/projected/33920f19-5306-4275-b275-3433be348592-kube-api-access-ssq4g\") pod \"33920f19-5306-4275-b275-3433be348592\" (UID: \"33920f19-5306-4275-b275-3433be348592\") "
Oct 30 22:05:20.970289 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:20.970231    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33920f19-5306-4275-b275-3433be348592-credential-socket" pod "33920f19-5306-4275-b275-3433be348592" (UID: "33920f19-5306-4275-b275-3433be348592"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:20.970460 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:20.970345    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/33920f19-5306-4275-b275-3433be348592-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33920f19-5306-4275-b275-3433be348592-istiod-ca-cert\") pod \"33920f19-5306-4275-b275-3433be348592\" (UID: \"33920f19-5306-4275-b275-3433be348592\") "
Oct 30 22:05:20.970460 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:20.970424    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33920f19-5306-4275-b275-3433be348592-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33920f19-5306-4275-b275-3433be348592-istio-data\") pod \"33920f19-5306-4275-b275-3433be348592\" (UID: \"33920f19-5306-4275-b275-3433be348592\") "
Oct 30 22:05:20.970567 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:20.970457    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33920f19-5306-4275-b275-3433be348592-workload-certs" pod "33920f19-5306-4275-b275-3433be348592" (UID: "33920f19-5306-4275-b275-3433be348592"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:20.970567 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:20.970487    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/33920f19-5306-4275-b275-3433be348592-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33920f19-5306-4275-b275-3433be348592-istio-podinfo\") pod \"33920f19-5306-4275-b275-3433be348592\" (UID: \"33920f19-5306-4275-b275-3433be348592\") "
Oct 30 22:05:20.971503 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:20.970794    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33920f19-5306-4275-b275-3433be348592-istio-data" pod "33920f19-5306-4275-b275-3433be348592" (UID: "33920f19-5306-4275-b275-3433be348592"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:20.971503 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:20.970801    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33920f19-5306-4275-b275-3433be348592-istiod-ca-cert" pod "33920f19-5306-4275-b275-3433be348592" (UID: "33920f19-5306-4275-b275-3433be348592"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:05:20.971503 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:20.971359    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33920f19-5306-4275-b275-3433be348592-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:20.971503 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:20.971415    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33920f19-5306-4275-b275-3433be348592-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:20.971503 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:20.971426    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33920f19-5306-4275-b275-3433be348592-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:20.971503 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:20.971436    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33920f19-5306-4275-b275-3433be348592-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:20.971941 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:20.971607    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33920f19-5306-4275-b275-3433be348592-workload-socket" pod "33920f19-5306-4275-b275-3433be348592" (UID: "33920f19-5306-4275-b275-3433be348592"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:20.973193 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:20.973164    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/33920f19-5306-4275-b275-3433be348592-istio-podinfo" pod "33920f19-5306-4275-b275-3433be348592" (UID: "33920f19-5306-4275-b275-3433be348592"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:05:20.973648 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:20.973625    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33920f19-5306-4275-b275-3433be348592-istio-envoy" pod "33920f19-5306-4275-b275-3433be348592" (UID: "33920f19-5306-4275-b275-3433be348592"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:20.973784 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:20.973761    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33920f19-5306-4275-b275-3433be348592-kube-api-access-ssq4g" pod "33920f19-5306-4275-b275-3433be348592" (UID: "33920f19-5306-4275-b275-3433be348592"). InnerVolumeSpecName "kube-api-access-ssq4g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:05:20.975696 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:20.975675    4871 generic.go:356] "Generic (PLEG): container finished" podID="33920f19-5306-4275-b275-3433be348592" containerID="903b49276da1b65dc699be10ae7773ed65cba1c379157ea01016890a10e3fb8b" exitCode=0
Oct 30 22:05:20.975764 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:20.975716    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"
Oct 30 22:05:20.975815 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:20.975757    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql" event={"ID":"33920f19-5306-4275-b275-3433be348592","Type":"ContainerDied","Data":"903b49276da1b65dc699be10ae7773ed65cba1c379157ea01016890a10e3fb8b"}
Oct 30 22:05:20.975872 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:20.975821    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql" event={"ID":"33920f19-5306-4275-b275-3433be348592","Type":"ContainerDied","Data":"4878d867837ff98711b5f3b2763678d42cbf569da6daaf889305d081d467cf16"}
Oct 30 22:05:20.975872 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:20.975858    4871 scope.go:122] "RemoveContainer" containerID="903b49276da1b65dc699be10ae7773ed65cba1c379157ea01016890a10e3fb8b"
Oct 30 22:05:20.977632 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:20.977613    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33920f19-5306-4275-b275-3433be348592-istio-token" pod "33920f19-5306-4275-b275-3433be348592" (UID: "33920f19-5306-4275-b275-3433be348592"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:05:21.033481 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:21.027171    4871 scope.go:122] "RemoveContainer" containerID="903b49276da1b65dc699be10ae7773ed65cba1c379157ea01016890a10e3fb8b"
Oct 30 22:05:21.033481 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:05:21.028627    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"903b49276da1b65dc699be10ae7773ed65cba1c379157ea01016890a10e3fb8b\": container with ID starting with 903b49276da1b65dc699be10ae7773ed65cba1c379157ea01016890a10e3fb8b not found: ID does not exist" containerID="903b49276da1b65dc699be10ae7773ed65cba1c379157ea01016890a10e3fb8b"
Oct 30 22:05:21.033481 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:21.028670    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"903b49276da1b65dc699be10ae7773ed65cba1c379157ea01016890a10e3fb8b"} err="failed to get container status \"903b49276da1b65dc699be10ae7773ed65cba1c379157ea01016890a10e3fb8b\": rpc error: code = NotFound desc = could not find container \"903b49276da1b65dc699be10ae7773ed65cba1c379157ea01016890a10e3fb8b\": container with ID starting with 903b49276da1b65dc699be10ae7773ed65cba1c379157ea01016890a10e3fb8b not found: ID does not exist"
Oct 30 22:05:21.073173 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:21.073095    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33920f19-5306-4275-b275-3433be348592-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:21.073173 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:21.073123    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33920f19-5306-4275-b275-3433be348592-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:21.073173 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:21.073133    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33920f19-5306-4275-b275-3433be348592-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:21.073173 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:21.073141    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33920f19-5306-4275-b275-3433be348592-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:21.073173 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:21.073150    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ssq4g\" (UniqueName: \"kubernetes.io/projected/33920f19-5306-4275-b275-3433be348592-kube-api-access-ssq4g\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:21.400730 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:21.400673    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"]
Oct 30 22:05:21.428569 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:21.428521    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-967ql"]
Oct 30 22:05:22.031267 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:22.028770    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="33920f19-5306-4275-b275-3433be348592" path="/var/lib/kubelet/pods/33920f19-5306-4275-b275-3433be348592/volumes"
Oct 30 22:05:24.002978 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:24.002941    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:26.258434 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:26.258393    4871 patch_prober.go:29] interesting pod/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:05:26.258842 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:26.258468    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4" podUID="722963e5-d33d-42a6-9f76-7c9b8d2b2fe0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:27.169078 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:27.169035    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"]
Oct 30 22:05:27.169458 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:27.169424    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq" podUID="84b9ee78-064c-4893-b81f-b9013b323448" containerName="istio-proxy" containerID="cri-o://f5735ba1aee7bc07b2251bca02fbcb206117a930dffc6ff7788239231f62d947" gracePeriod=30
Oct 30 22:05:27.213993 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:27.213952    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"]
Oct 30 22:05:27.214591 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:27.214528    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd" podUID="5f3ddcc7-e832-42e9-97d4-1541f5a09044" containerName="istio-proxy" containerID="cri-o://b7cc2da8b56f98b3003f2c936c4c963d28a532ad010e7069ae24ec67b74aa620" gracePeriod=30
Oct 30 22:05:27.219886 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:27.219858    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"]
Oct 30 22:05:27.220073 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:27.220051    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj" podUID="d9a09763-b456-469d-80c7-a787456003f4" containerName="istio-proxy" containerID="cri-o://683650a931b7ff26ab034048cd5f25a91bb183c0453af8191f7c446ae45fc24c" gracePeriod=30
Oct 30 22:05:27.264162 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:27.264095    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"]
Oct 30 22:05:27.265443 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:27.265404    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs" podUID="78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd" containerName="istio-proxy" containerID="cri-o://b470061233344cef6995cfb7ed13d4b9b8096dfba147259a4963308212df1ca3" gracePeriod=30
Oct 30 22:05:27.717479 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:27.717441    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:28.653283 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:28.653230    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:28.866451 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:28.866395    4871 patch_prober.go:29] interesting pod/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:05:28.866652 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:28.866486    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l" podUID="d5e633ba-5844-4f32-956d-8757ce76acce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:29.168213 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:29.168177    4871 generic.go:356] "Generic (PLEG): container finished" podID="d5e633ba-5844-4f32-956d-8757ce76acce" containerID="be36c755fc6a62139fac349c74f708a0df0de657e27ccd05b45943253d316aae" exitCode=0
Oct 30 22:05:29.168453 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:29.168289    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l" event={"ID":"d5e633ba-5844-4f32-956d-8757ce76acce","Type":"ContainerDied","Data":"be36c755fc6a62139fac349c74f708a0df0de657e27ccd05b45943253d316aae"}
Oct 30 22:05:29.168804 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:29.168781    4871 scope.go:122] "RemoveContainer" containerID="be36c755fc6a62139fac349c74f708a0df0de657e27ccd05b45943253d316aae"
Oct 30 22:05:29.170311 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:29.170280    4871 generic.go:356] "Generic (PLEG): container finished" podID="722963e5-d33d-42a6-9f76-7c9b8d2b2fe0" containerID="96f88fa306b5954d40cf5ef622ef33aadbe249cda9dbd69cb5452f869563ba45" exitCode=0
Oct 30 22:05:29.170427 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:29.170335    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4" event={"ID":"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0","Type":"ContainerDied","Data":"96f88fa306b5954d40cf5ef622ef33aadbe249cda9dbd69cb5452f869563ba45"}
Oct 30 22:05:29.170745 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:29.170717    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:05:29.170812 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:29.170770    4871 scope.go:122] "RemoveContainer" containerID="96f88fa306b5954d40cf5ef622ef33aadbe249cda9dbd69cb5452f869563ba45"
Oct 30 22:05:29.171036 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:29.170806    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:05:29.171036 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:29.170853    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:05:29.172298 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:29.172273    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:05:29.172360 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:29.172337    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:05:29.172399 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:29.172366    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:05:29.530838 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:29.530651    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:05:29.530838 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:29.530722    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj" podUID="d9a09763-b456-469d-80c7-a787456003f4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:05:30.181175 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:30.181136    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l" event={"ID":"d5e633ba-5844-4f32-956d-8757ce76acce","Type":"ContainerStarted","Data":"f30ddb4ace4b9375945ae44f718c1761e69dfb42111929ccac7ab160d117e158"}
Oct 30 22:05:30.183387 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:30.183360    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4" event={"ID":"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0","Type":"ContainerStarted","Data":"9c0c58dc21db3aa8e2a7a12aba5cf75b7672a1edb6f583cefcd5823cb2abef8c"}
Oct 30 22:05:30.553719 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:30.553622    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"]
Oct 30 22:05:30.554173 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:30.554140    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh" podUID="47d9d464-c7c2-4980-ba4d-e3320373ed88" containerName="istio-proxy" containerID="cri-o://88cce796c963ab6a228d9846d6b5114217db53e246ee8de0a3c3871a35e3cfbc" gracePeriod=30
Oct 30 22:05:30.566100 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:30.566078    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"
Oct 30 22:05:30.566375 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:30.566357    4871 patch_prober.go:29] interesting pod/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 30 22:05:30.566510 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:30.566493    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l" podUID="d5e633ba-5844-4f32-956d-8757ce76acce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 30 22:05:30.792776 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:30.792735    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"
Oct 30 22:05:30.793399 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:30.793371    4871 patch_prober.go:29] interesting pod/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 30 22:05:30.793606 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:30.793581    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4" podUID="722963e5-d33d-42a6-9f76-7c9b8d2b2fe0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 30 22:05:31.566503 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:31.566465    4871 patch_prober.go:29] interesting pod/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 30 22:05:31.566969 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:31.566536    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l" podUID="d5e633ba-5844-4f32-956d-8757ce76acce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 30 22:05:31.786592 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:31.786552    4871 patch_prober.go:29] interesting pod/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 30 22:05:31.786827 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:31.786630    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4" podUID="722963e5-d33d-42a6-9f76-7c9b8d2b2fe0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 30 22:05:32.584171 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:32.583739    4871 patch_prober.go:29] interesting pod/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 30 22:05:32.584171 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:32.583799    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l" podUID="d5e633ba-5844-4f32-956d-8757ce76acce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 30 22:05:32.794041 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:32.794001    4871 patch_prober.go:29] interesting pod/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 30 22:05:32.794224 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:32.794062    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4" podUID="722963e5-d33d-42a6-9f76-7c9b8d2b2fe0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 30 22:05:33.016468 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:32.997453    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"
Oct 30 22:05:33.136292 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.126069    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84b9ee78-064c-4893-b81f-b9013b323448-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84b9ee78-064c-4893-b81f-b9013b323448-credential-socket\") pod \"84b9ee78-064c-4893-b81f-b9013b323448\" (UID: \"84b9ee78-064c-4893-b81f-b9013b323448\") "
Oct 30 22:05:33.137043 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.137010    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84b9ee78-064c-4893-b81f-b9013b323448-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84b9ee78-064c-4893-b81f-b9013b323448-workload-certs\") pod \"84b9ee78-064c-4893-b81f-b9013b323448\" (UID: \"84b9ee78-064c-4893-b81f-b9013b323448\") "
Oct 30 22:05:33.137297 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.137277    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/84b9ee78-064c-4893-b81f-b9013b323448-istio-token\" (UniqueName: \"kubernetes.io/projected/84b9ee78-064c-4893-b81f-b9013b323448-istio-token\") pod \"84b9ee78-064c-4893-b81f-b9013b323448\" (UID: \"84b9ee78-064c-4893-b81f-b9013b323448\") "
Oct 30 22:05:33.137493 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.137468    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/84b9ee78-064c-4893-b81f-b9013b323448-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84b9ee78-064c-4893-b81f-b9013b323448-istiod-ca-cert\") pod \"84b9ee78-064c-4893-b81f-b9013b323448\" (UID: \"84b9ee78-064c-4893-b81f-b9013b323448\") "
Oct 30 22:05:33.137643 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.137628    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84b9ee78-064c-4893-b81f-b9013b323448-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84b9ee78-064c-4893-b81f-b9013b323448-workload-socket\") pod \"84b9ee78-064c-4893-b81f-b9013b323448\" (UID: \"84b9ee78-064c-4893-b81f-b9013b323448\") "
Oct 30 22:05:33.137791 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.137775    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/84b9ee78-064c-4893-b81f-b9013b323448-kube-api-access-99h6f\" (UniqueName: \"kubernetes.io/projected/84b9ee78-064c-4893-b81f-b9013b323448-kube-api-access-99h6f\") pod \"84b9ee78-064c-4893-b81f-b9013b323448\" (UID: \"84b9ee78-064c-4893-b81f-b9013b323448\") "
Oct 30 22:05:33.137914 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.137900    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/84b9ee78-064c-4893-b81f-b9013b323448-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84b9ee78-064c-4893-b81f-b9013b323448-istio-podinfo\") pod \"84b9ee78-064c-4893-b81f-b9013b323448\" (UID: \"84b9ee78-064c-4893-b81f-b9013b323448\") "
Oct 30 22:05:33.138030 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.138017    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84b9ee78-064c-4893-b81f-b9013b323448-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84b9ee78-064c-4893-b81f-b9013b323448-istio-envoy\") pod \"84b9ee78-064c-4893-b81f-b9013b323448\" (UID: \"84b9ee78-064c-4893-b81f-b9013b323448\") "
Oct 30 22:05:33.138139 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.138126    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84b9ee78-064c-4893-b81f-b9013b323448-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84b9ee78-064c-4893-b81f-b9013b323448-istio-data\") pod \"84b9ee78-064c-4893-b81f-b9013b323448\" (UID: \"84b9ee78-064c-4893-b81f-b9013b323448\") "
Oct 30 22:05:33.139379 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.136847    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84b9ee78-064c-4893-b81f-b9013b323448-credential-socket" pod "84b9ee78-064c-4893-b81f-b9013b323448" (UID: "84b9ee78-064c-4893-b81f-b9013b323448"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:33.271365 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.139354    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84b9ee78-064c-4893-b81f-b9013b323448-istio-data" pod "84b9ee78-064c-4893-b81f-b9013b323448" (UID: "84b9ee78-064c-4893-b81f-b9013b323448"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:33.271365 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.154143    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84b9ee78-064c-4893-b81f-b9013b323448-workload-socket" pod "84b9ee78-064c-4893-b81f-b9013b323448" (UID: "84b9ee78-064c-4893-b81f-b9013b323448"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:33.271365 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.154377    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84b9ee78-064c-4893-b81f-b9013b323448-workload-certs" pod "84b9ee78-064c-4893-b81f-b9013b323448" (UID: "84b9ee78-064c-4893-b81f-b9013b323448"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:33.271365 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.258594    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84b9ee78-064c-4893-b81f-b9013b323448-istiod-ca-cert" pod "84b9ee78-064c-4893-b81f-b9013b323448" (UID: "84b9ee78-064c-4893-b81f-b9013b323448"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:05:33.309544 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.294946    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84b9ee78-064c-4893-b81f-b9013b323448-kube-api-access-99h6f" pod "84b9ee78-064c-4893-b81f-b9013b323448" (UID: "84b9ee78-064c-4893-b81f-b9013b323448"). InnerVolumeSpecName "kube-api-access-99h6f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:05:33.309982 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.309835    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84b9ee78-064c-4893-b81f-b9013b323448-istio-token" pod "84b9ee78-064c-4893-b81f-b9013b323448" (UID: "84b9ee78-064c-4893-b81f-b9013b323448"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:05:33.309982 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.309932    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/84b9ee78-064c-4893-b81f-b9013b323448-istio-podinfo" pod "84b9ee78-064c-4893-b81f-b9013b323448" (UID: "84b9ee78-064c-4893-b81f-b9013b323448"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:05:33.407262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.391112    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84b9ee78-064c-4893-b81f-b9013b323448-istio-envoy" pod "84b9ee78-064c-4893-b81f-b9013b323448" (UID: "84b9ee78-064c-4893-b81f-b9013b323448"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:33.534951 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.509038    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84b9ee78-064c-4893-b81f-b9013b323448-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:33.534951 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.509081    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84b9ee78-064c-4893-b81f-b9013b323448-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:33.534951 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.509096    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84b9ee78-064c-4893-b81f-b9013b323448-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:33.534951 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.509118    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84b9ee78-064c-4893-b81f-b9013b323448-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:33.534951 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.509132    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84b9ee78-064c-4893-b81f-b9013b323448-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:33.534951 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.509148    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-99h6f\" (UniqueName: \"kubernetes.io/projected/84b9ee78-064c-4893-b81f-b9013b323448-kube-api-access-99h6f\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:33.534951 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.509163    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84b9ee78-064c-4893-b81f-b9013b323448-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:33.534951 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.509181    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84b9ee78-064c-4893-b81f-b9013b323448-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:33.534951 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.509194    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84b9ee78-064c-4893-b81f-b9013b323448-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:33.687395 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.687355    4871 patch_prober.go:29] interesting pod/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 30 22:05:33.688696 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.687965    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l" podUID="d5e633ba-5844-4f32-956d-8757ce76acce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 30 22:05:33.857263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.846930    4871 patch_prober.go:29] interesting pod/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 30 22:05:33.857263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.847005    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4" podUID="722963e5-d33d-42a6-9f76-7c9b8d2b2fe0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 30 22:05:33.882604 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.882566    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"
Oct 30 22:05:33.969603 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.946559    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9a09763-b456-469d-80c7-a787456003f4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a09763-b456-469d-80c7-a787456003f4-workload-socket\") pod \"d9a09763-b456-469d-80c7-a787456003f4\" (UID: \"d9a09763-b456-469d-80c7-a787456003f4\") "
Oct 30 22:05:33.969925 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.969890    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d9a09763-b456-469d-80c7-a787456003f4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9a09763-b456-469d-80c7-a787456003f4-istio-podinfo\") pod \"d9a09763-b456-469d-80c7-a787456003f4\" (UID: \"d9a09763-b456-469d-80c7-a787456003f4\") "
Oct 30 22:05:33.970132 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.970114    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d9a09763-b456-469d-80c7-a787456003f4-istio-token\" (UniqueName: \"kubernetes.io/projected/d9a09763-b456-469d-80c7-a787456003f4-istio-token\") pod \"d9a09763-b456-469d-80c7-a787456003f4\" (UID: \"d9a09763-b456-469d-80c7-a787456003f4\") "
Oct 30 22:05:33.970376 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.970356    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9a09763-b456-469d-80c7-a787456003f4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9a09763-b456-469d-80c7-a787456003f4-workload-certs\") pod \"d9a09763-b456-469d-80c7-a787456003f4\" (UID: \"d9a09763-b456-469d-80c7-a787456003f4\") "
Oct 30 22:05:33.970625 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.970609    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d9a09763-b456-469d-80c7-a787456003f4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9a09763-b456-469d-80c7-a787456003f4-istiod-ca-cert\") pod \"d9a09763-b456-469d-80c7-a787456003f4\" (UID: \"d9a09763-b456-469d-80c7-a787456003f4\") "
Oct 30 22:05:33.970798 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.970785    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9a09763-b456-469d-80c7-a787456003f4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9a09763-b456-469d-80c7-a787456003f4-istio-data\") pod \"d9a09763-b456-469d-80c7-a787456003f4\" (UID: \"d9a09763-b456-469d-80c7-a787456003f4\") "
Oct 30 22:05:33.970922 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.970907    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d9a09763-b456-469d-80c7-a787456003f4-kube-api-access-t6rsf\" (UniqueName: \"kubernetes.io/projected/d9a09763-b456-469d-80c7-a787456003f4-kube-api-access-t6rsf\") pod \"d9a09763-b456-469d-80c7-a787456003f4\" (UID: \"d9a09763-b456-469d-80c7-a787456003f4\") "
Oct 30 22:05:33.971028 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.971016    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9a09763-b456-469d-80c7-a787456003f4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9a09763-b456-469d-80c7-a787456003f4-istio-envoy\") pod \"d9a09763-b456-469d-80c7-a787456003f4\" (UID: \"d9a09763-b456-469d-80c7-a787456003f4\") "
Oct 30 22:05:33.971227 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.971214    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9a09763-b456-469d-80c7-a787456003f4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a09763-b456-469d-80c7-a787456003f4-credential-socket\") pod \"d9a09763-b456-469d-80c7-a787456003f4\" (UID: \"d9a09763-b456-469d-80c7-a787456003f4\") "
Oct 30 22:05:34.007276 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.973351    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9a09763-b456-469d-80c7-a787456003f4-workload-socket" pod "d9a09763-b456-469d-80c7-a787456003f4" (UID: "d9a09763-b456-469d-80c7-a787456003f4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:34.007276 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:33.996724    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9a09763-b456-469d-80c7-a787456003f4-istio-data" pod "d9a09763-b456-469d-80c7-a787456003f4" (UID: "d9a09763-b456-469d-80c7-a787456003f4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:34.009548 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.009481    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9a09763-b456-469d-80c7-a787456003f4-credential-socket" pod "d9a09763-b456-469d-80c7-a787456003f4" (UID: "d9a09763-b456-469d-80c7-a787456003f4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:34.043332 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.021611    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9a09763-b456-469d-80c7-a787456003f4-workload-certs" pod "d9a09763-b456-469d-80c7-a787456003f4" (UID: "d9a09763-b456-469d-80c7-a787456003f4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:34.043332 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.021784    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9a09763-b456-469d-80c7-a787456003f4-istiod-ca-cert" pod "d9a09763-b456-469d-80c7-a787456003f4" (UID: "d9a09763-b456-469d-80c7-a787456003f4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:05:34.102528 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.102489    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:34.227295 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.129691    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a09763-b456-469d-80c7-a787456003f4-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:34.227295 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.129733    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a09763-b456-469d-80c7-a787456003f4-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:34.227295 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.129751    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9a09763-b456-469d-80c7-a787456003f4-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:34.227295 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.129774    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9a09763-b456-469d-80c7-a787456003f4-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:34.227295 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.129789    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9a09763-b456-469d-80c7-a787456003f4-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:34.227295 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.211810    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d9a09763-b456-469d-80c7-a787456003f4-istio-podinfo" pod "d9a09763-b456-469d-80c7-a787456003f4" (UID: "d9a09763-b456-469d-80c7-a787456003f4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:05:34.267589 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.267520    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9a09763-b456-469d-80c7-a787456003f4-istio-token" pod "d9a09763-b456-469d-80c7-a787456003f4" (UID: "d9a09763-b456-469d-80c7-a787456003f4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:05:34.299311 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.289732    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9a09763-b456-469d-80c7-a787456003f4-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:34.299311 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.289781    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9a09763-b456-469d-80c7-a787456003f4-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:34.317266 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.312890    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9a09763-b456-469d-80c7-a787456003f4-istio-envoy" pod "d9a09763-b456-469d-80c7-a787456003f4" (UID: "d9a09763-b456-469d-80c7-a787456003f4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:34.317266 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.313085    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9a09763-b456-469d-80c7-a787456003f4-kube-api-access-t6rsf" pod "d9a09763-b456-469d-80c7-a787456003f4" (UID: "d9a09763-b456-469d-80c7-a787456003f4"). InnerVolumeSpecName "kube-api-access-t6rsf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:05:34.338663 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.337839    4871 generic.go:356] "Generic (PLEG): container finished" podID="84b9ee78-064c-4893-b81f-b9013b323448" containerID="f5735ba1aee7bc07b2251bca02fbcb206117a930dffc6ff7788239231f62d947" exitCode=0
Oct 30 22:05:34.338663 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.338089    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"
Oct 30 22:05:34.404341 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.395181    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"
Oct 30 22:05:34.408272 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.404980    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"
Oct 30 22:05:34.408272 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.406771    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6rsf\" (UniqueName: \"kubernetes.io/projected/d9a09763-b456-469d-80c7-a787456003f4-kube-api-access-t6rsf\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:34.408272 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.406809    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9a09763-b456-469d-80c7-a787456003f4-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:34.423906 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.423859    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq" event={"ID":"84b9ee78-064c-4893-b81f-b9013b323448","Type":"ContainerDied","Data":"f5735ba1aee7bc07b2251bca02fbcb206117a930dffc6ff7788239231f62d947"}
Oct 30 22:05:34.438070 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.438038    4871 scope.go:122] "RemoveContainer" containerID="f5735ba1aee7bc07b2251bca02fbcb206117a930dffc6ff7788239231f62d947"
Oct 30 22:05:34.438710 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.438586    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq" event={"ID":"84b9ee78-064c-4893-b81f-b9013b323448","Type":"ContainerDied","Data":"1d4712689e481871c4df97e3a3e51beb73f20711713f00e33d72ee8edc9305be"}
Oct 30 22:05:34.461636 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.461589    4871 generic.go:356] "Generic (PLEG): container finished" podID="5f3ddcc7-e832-42e9-97d4-1541f5a09044" containerID="b7cc2da8b56f98b3003f2c936c4c963d28a532ad010e7069ae24ec67b74aa620" exitCode=0
Oct 30 22:05:34.461850 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.461748    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd" event={"ID":"5f3ddcc7-e832-42e9-97d4-1541f5a09044","Type":"ContainerDied","Data":"b7cc2da8b56f98b3003f2c936c4c963d28a532ad010e7069ae24ec67b74aa620"}
Oct 30 22:05:34.478180 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.478130    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.94:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:05:34.478495 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.478449    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs" podUID="78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.94:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:34.508274 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.492847    4871 generic.go:356] "Generic (PLEG): container finished" podID="d9a09763-b456-469d-80c7-a787456003f4" containerID="683650a931b7ff26ab034048cd5f25a91bb183c0453af8191f7c446ae45fc24c" exitCode=0
Oct 30 22:05:34.508274 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.492965    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj" event={"ID":"d9a09763-b456-469d-80c7-a787456003f4","Type":"ContainerDied","Data":"683650a931b7ff26ab034048cd5f25a91bb183c0453af8191f7c446ae45fc24c"}
Oct 30 22:05:34.508274 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.493019    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj" event={"ID":"d9a09763-b456-469d-80c7-a787456003f4","Type":"ContainerDied","Data":"7540e858f52ed3575b6e1a9aca9788822210b98b1040f4a57f4c3603c147bd37"}
Oct 30 22:05:34.508274 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.493107    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"
Oct 30 22:05:34.514684 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.514653    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-istio-envoy\") pod \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\" (UID: \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\") "
Oct 30 22:05:34.514966 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.514947    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5f3ddcc7-e832-42e9-97d4-1541f5a09044-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f3ddcc7-e832-42e9-97d4-1541f5a09044-istiod-ca-cert\") pod \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\" (UID: \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\") "
Oct 30 22:05:34.515681 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.515658    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-kube-api-access-wmlvj\" (UniqueName: \"kubernetes.io/projected/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-kube-api-access-wmlvj\") pod \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\" (UID: \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\") "
Oct 30 22:05:34.516015 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.515999    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-istiod-ca-cert\") pod \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\" (UID: \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\") "
Oct 30 22:05:34.549407 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.515540    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f3ddcc7-e832-42e9-97d4-1541f5a09044-istiod-ca-cert" pod "5f3ddcc7-e832-42e9-97d4-1541f5a09044" (UID: "5f3ddcc7-e832-42e9-97d4-1541f5a09044"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:05:34.554319 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.522062    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-istiod-ca-cert" pod "78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd" (UID: "78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:05:34.554513 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.550650    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f3ddcc7-e832-42e9-97d4-1541f5a09044-workload-socket" pod "5f3ddcc7-e832-42e9-97d4-1541f5a09044" (UID: "5f3ddcc7-e832-42e9-97d4-1541f5a09044"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:34.554609 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.550929    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs" event={"ID":"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd","Type":"ContainerDied","Data":"b470061233344cef6995cfb7ed13d4b9b8096dfba147259a4963308212df1ca3"}
Oct 30 22:05:34.554992 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.550999    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"
Oct 30 22:05:34.555359 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.549005    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5f3ddcc7-e832-42e9-97d4-1541f5a09044-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f3ddcc7-e832-42e9-97d4-1541f5a09044-workload-socket\") pod \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\" (UID: \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\") "
Oct 30 22:05:34.555587 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.555573    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5f3ddcc7-e832-42e9-97d4-1541f5a09044-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f3ddcc7-e832-42e9-97d4-1541f5a09044-workload-certs\") pod \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\" (UID: \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\") "
Oct 30 22:05:34.555745 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.555729    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5f3ddcc7-e832-42e9-97d4-1541f5a09044-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f3ddcc7-e832-42e9-97d4-1541f5a09044-istio-podinfo\") pod \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\" (UID: \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\") "
Oct 30 22:05:34.555891 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.555866    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-workload-socket\") pod \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\" (UID: \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\") "
Oct 30 22:05:34.556010 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.555998    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-istio-data\") pod \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\" (UID: \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\") "
Oct 30 22:05:34.556149 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.556137    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-istio-token\" (UniqueName: \"kubernetes.io/projected/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-istio-token\") pod \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\" (UID: \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\") "
Oct 30 22:05:34.556311 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.556298    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-workload-certs\") pod \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\" (UID: \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\") "
Oct 30 22:05:34.612622 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.592325    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5f3ddcc7-e832-42e9-97d4-1541f5a09044-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f3ddcc7-e832-42e9-97d4-1541f5a09044-istio-envoy\") pod \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\" (UID: \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\") "
Oct 30 22:05:34.612622 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.592391    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5f3ddcc7-e832-42e9-97d4-1541f5a09044-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f3ddcc7-e832-42e9-97d4-1541f5a09044-credential-socket\") pod \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\" (UID: \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\") "
Oct 30 22:05:34.612622 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.592545    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5f3ddcc7-e832-42e9-97d4-1541f5a09044-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f3ddcc7-e832-42e9-97d4-1541f5a09044-istio-data\") pod \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\" (UID: \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\") "
Oct 30 22:05:34.612622 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.592583    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5f3ddcc7-e832-42e9-97d4-1541f5a09044-kube-api-access-nd4kq\" (UniqueName: \"kubernetes.io/projected/5f3ddcc7-e832-42e9-97d4-1541f5a09044-kube-api-access-nd4kq\") pod \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\" (UID: \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\") "
Oct 30 22:05:34.612622 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.592635    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-credential-socket\") pod \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\" (UID: \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\") "
Oct 30 22:05:34.612622 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.592706    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5f3ddcc7-e832-42e9-97d4-1541f5a09044-istio-token\" (UniqueName: \"kubernetes.io/projected/5f3ddcc7-e832-42e9-97d4-1541f5a09044-istio-token\") pod \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\" (UID: \"5f3ddcc7-e832-42e9-97d4-1541f5a09044\") "
Oct 30 22:05:34.612622 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.592792    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-istio-podinfo\") pod \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\" (UID: \"78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd\") "
Oct 30 22:05:34.612622 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.593586    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f3ddcc7-e832-42e9-97d4-1541f5a09044-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:34.612622 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.593612    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:34.612622 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.593627    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f3ddcc7-e832-42e9-97d4-1541f5a09044-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:34.612622 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.566470    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-istio-envoy" pod "78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd" (UID: "78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:34.612622 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.574627    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-workload-socket" pod "78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd" (UID: "78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:34.612622 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.576308    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-workload-certs" pod "78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd" (UID: "78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:34.612622 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.576387    4871 patch_prober.go:29] interesting pod/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 30 22:05:34.612622 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.595565    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l" podUID="d5e633ba-5844-4f32-956d-8757ce76acce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 30 22:05:34.612622 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.601377    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f3ddcc7-e832-42e9-97d4-1541f5a09044-credential-socket" pod "5f3ddcc7-e832-42e9-97d4-1541f5a09044" (UID: "5f3ddcc7-e832-42e9-97d4-1541f5a09044"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:34.619003 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.618966    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f3ddcc7-e832-42e9-97d4-1541f5a09044-istio-data" pod "5f3ddcc7-e832-42e9-97d4-1541f5a09044" (UID: "5f3ddcc7-e832-42e9-97d4-1541f5a09044"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:34.619405 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.619382    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-credential-socket" pod "78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd" (UID: "78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:34.629270 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.550905    4871 generic.go:356] "Generic (PLEG): container finished" podID="78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd" containerID="b470061233344cef6995cfb7ed13d4b9b8096dfba147259a4963308212df1ca3" exitCode=0
Oct 30 22:05:34.629270 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.577585    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f3ddcc7-e832-42e9-97d4-1541f5a09044-workload-certs" pod "5f3ddcc7-e832-42e9-97d4-1541f5a09044" (UID: "5f3ddcc7-e832-42e9-97d4-1541f5a09044"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:34.629270 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.577925    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-istio-data" pod "78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd" (UID: "78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:34.629270 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.589861    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-kube-api-access-wmlvj" pod "78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd" (UID: "78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd"). InnerVolumeSpecName "kube-api-access-wmlvj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:05:34.631755 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.630377    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5f3ddcc7-e832-42e9-97d4-1541f5a09044-istio-podinfo" pod "5f3ddcc7-e832-42e9-97d4-1541f5a09044" (UID: "5f3ddcc7-e832-42e9-97d4-1541f5a09044"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:05:34.682434 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.680975    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f3ddcc7-e832-42e9-97d4-1541f5a09044-istio-envoy" pod "5f3ddcc7-e832-42e9-97d4-1541f5a09044" (UID: "5f3ddcc7-e832-42e9-97d4-1541f5a09044"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:34.682434 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.682095    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f3ddcc7-e832-42e9-97d4-1541f5a09044-istio-token" pod "5f3ddcc7-e832-42e9-97d4-1541f5a09044" (UID: "5f3ddcc7-e832-42e9-97d4-1541f5a09044"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:05:34.683266 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.683219    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-istio-podinfo" pod "78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd" (UID: "78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:05:34.685837 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.685797    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f3ddcc7-e832-42e9-97d4-1541f5a09044-kube-api-access-nd4kq" pod "5f3ddcc7-e832-42e9-97d4-1541f5a09044" (UID: "5f3ddcc7-e832-42e9-97d4-1541f5a09044"). InnerVolumeSpecName "kube-api-access-nd4kq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:05:34.686111 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.686076    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-istio-token" pod "78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd" (UID: "78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:05:34.784880 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.783091    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f3ddcc7-e832-42e9-97d4-1541f5a09044-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:34.784880 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.783133    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f3ddcc7-e832-42e9-97d4-1541f5a09044-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:34.784880 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.783148    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:34.784880 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.783162    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:34.784880 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.783176    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:34.784880 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.783190    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:34.784880 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.783206    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f3ddcc7-e832-42e9-97d4-1541f5a09044-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:34.784880 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.783220    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f3ddcc7-e832-42e9-97d4-1541f5a09044-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:34.784880 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.783248    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f3ddcc7-e832-42e9-97d4-1541f5a09044-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:34.784880 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.783263    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nd4kq\" (UniqueName: \"kubernetes.io/projected/5f3ddcc7-e832-42e9-97d4-1541f5a09044-kube-api-access-nd4kq\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:34.784880 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.783279    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:34.784880 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.783291    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f3ddcc7-e832-42e9-97d4-1541f5a09044-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:34.788287 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.783302    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:34.788287 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.786064    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:34.788287 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.786087    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmlvj\" (UniqueName: \"kubernetes.io/projected/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd-kube-api-access-wmlvj\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:34.788287 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.786839    4871 patch_prober.go:29] interesting pod/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 30 22:05:34.788287 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.786888    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4" podUID="722963e5-d33d-42a6-9f76-7c9b8d2b2fe0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 30 22:05:34.876883 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.876044    4871 scope.go:122] "RemoveContainer" containerID="f5735ba1aee7bc07b2251bca02fbcb206117a930dffc6ff7788239231f62d947"
Oct 30 22:05:34.893564 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:05:34.893479    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5735ba1aee7bc07b2251bca02fbcb206117a930dffc6ff7788239231f62d947\": container with ID starting with f5735ba1aee7bc07b2251bca02fbcb206117a930dffc6ff7788239231f62d947 not found: ID does not exist" containerID="f5735ba1aee7bc07b2251bca02fbcb206117a930dffc6ff7788239231f62d947"
Oct 30 22:05:34.893758 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.893572    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5735ba1aee7bc07b2251bca02fbcb206117a930dffc6ff7788239231f62d947"} err="failed to get container status \"f5735ba1aee7bc07b2251bca02fbcb206117a930dffc6ff7788239231f62d947\": rpc error: code = NotFound desc = could not find container \"f5735ba1aee7bc07b2251bca02fbcb206117a930dffc6ff7788239231f62d947\": container with ID starting with f5735ba1aee7bc07b2251bca02fbcb206117a930dffc6ff7788239231f62d947 not found: ID does not exist"
Oct 30 22:05:34.893758 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.893603    4871 scope.go:122] "RemoveContainer" containerID="b7cc2da8b56f98b3003f2c936c4c963d28a532ad010e7069ae24ec67b74aa620"
Oct 30 22:05:34.964217 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.963774    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"]
Oct 30 22:05:34.983851 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:34.983809    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-l8hdj"]
Oct 30 22:05:35.082647 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:35.082593    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"]
Oct 30 22:05:35.082867 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:35.082672    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-f7gzq"]
Oct 30 22:05:35.099278 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:05:35.096752    4871 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78dcf4bb_0d68_4cd6_b280_0b8e070a4bcd.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78dcf4bb_0d68_4cd6_b280_0b8e070a4bcd.slice/crio-9e83ba591d7bfae9edad8e9231b2b1cb1b29303921780664380bb2c18759bdc7\": RecentStats: unable to find data in memory cache]"
Oct 30 22:05:35.211704 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:35.185463    4871 scope.go:122] "RemoveContainer" containerID="683650a931b7ff26ab034048cd5f25a91bb183c0453af8191f7c446ae45fc24c"
Oct 30 22:05:35.253320 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:35.253287    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"]
Oct 30 22:05:35.316262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:35.301770    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-n2vcs"]
Oct 30 22:05:35.353259 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:35.337407    4871 scope.go:122] "RemoveContainer" containerID="683650a931b7ff26ab034048cd5f25a91bb183c0453af8191f7c446ae45fc24c"
Oct 30 22:05:35.353259 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:05:35.337935    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"683650a931b7ff26ab034048cd5f25a91bb183c0453af8191f7c446ae45fc24c\": container with ID starting with 683650a931b7ff26ab034048cd5f25a91bb183c0453af8191f7c446ae45fc24c not found: ID does not exist" containerID="683650a931b7ff26ab034048cd5f25a91bb183c0453af8191f7c446ae45fc24c"
Oct 30 22:05:35.353259 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:35.337974    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"683650a931b7ff26ab034048cd5f25a91bb183c0453af8191f7c446ae45fc24c"} err="failed to get container status \"683650a931b7ff26ab034048cd5f25a91bb183c0453af8191f7c446ae45fc24c\": rpc error: code = NotFound desc = could not find container \"683650a931b7ff26ab034048cd5f25a91bb183c0453af8191f7c446ae45fc24c\": container with ID starting with 683650a931b7ff26ab034048cd5f25a91bb183c0453af8191f7c446ae45fc24c not found: ID does not exist"
Oct 30 22:05:35.353259 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:35.338000    4871 scope.go:122] "RemoveContainer" containerID="b470061233344cef6995cfb7ed13d4b9b8096dfba147259a4963308212df1ca3"
Oct 30 22:05:35.576816 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:35.566630    4871 patch_prober.go:29] interesting pod/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 30 22:05:35.576816 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:35.566698    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l" podUID="d5e633ba-5844-4f32-956d-8757ce76acce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 30 22:05:35.742618 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:05:35.742551    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:05:35.763265 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:35.759548    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd" event={"ID":"5f3ddcc7-e832-42e9-97d4-1541f5a09044","Type":"ContainerDied","Data":"7c0cb35df4e8fd10fd40d9d717ab67c7320af06814263396b5640586d2c31f37"}
Oct 30 22:05:35.763265 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:35.759693    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"
Oct 30 22:05:35.800017 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:35.799298    4871 patch_prober.go:29] interesting pod/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 30 22:05:35.800017 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:35.799366    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4" podUID="722963e5-d33d-42a6-9f76-7c9b8d2b2fe0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 30 22:05:36.023638 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:36.023600    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:36.281261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:36.248537    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd" path="/var/lib/kubelet/pods/78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd/volumes"
Oct 30 22:05:36.341318 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:36.333182    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="84b9ee78-064c-4893-b81f-b9013b323448" path="/var/lib/kubelet/pods/84b9ee78-064c-4893-b81f-b9013b323448/volumes"
Oct 30 22:05:36.341318 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:36.334063    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d9a09763-b456-469d-80c7-a787456003f4" path="/var/lib/kubelet/pods/d9a09763-b456-469d-80c7-a787456003f4/volumes"
Oct 30 22:05:36.341318 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:36.334555    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"]
Oct 30 22:05:36.341318 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:36.290842    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:36.412321 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:36.412280    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-44nfd"]
Oct 30 22:05:36.590326 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:36.570628    4871 patch_prober.go:29] interesting pod/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 30 22:05:36.590617 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:36.590573    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l" podUID="d5e633ba-5844-4f32-956d-8757ce76acce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 30 22:05:36.786847 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:36.786803    4871 patch_prober.go:29] interesting pod/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 30 22:05:36.787027 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:36.786874    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4" podUID="722963e5-d33d-42a6-9f76-7c9b8d2b2fe0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 30 22:05:36.873699 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:36.873665    4871 generic.go:356] "Generic (PLEG): container finished" podID="47d9d464-c7c2-4980-ba4d-e3320373ed88" containerID="88cce796c963ab6a228d9846d6b5114217db53e246ee8de0a3c3871a35e3cfbc" exitCode=0
Oct 30 22:05:36.874150 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:36.873951    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh" event={"ID":"47d9d464-c7c2-4980-ba4d-e3320373ed88","Type":"ContainerDied","Data":"88cce796c963ab6a228d9846d6b5114217db53e246ee8de0a3c3871a35e3cfbc"}
Oct 30 22:05:36.981602 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:36.981568    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"
Oct 30 22:05:37.152847 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:37.152748    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47d9d464-c7c2-4980-ba4d-e3320373ed88-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47d9d464-c7c2-4980-ba4d-e3320373ed88-credential-socket\") pod \"47d9d464-c7c2-4980-ba4d-e3320373ed88\" (UID: \"47d9d464-c7c2-4980-ba4d-e3320373ed88\") "
Oct 30 22:05:37.153655 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:37.153630    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47d9d464-c7c2-4980-ba4d-e3320373ed88-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47d9d464-c7c2-4980-ba4d-e3320373ed88-workload-socket\") pod \"47d9d464-c7c2-4980-ba4d-e3320373ed88\" (UID: \"47d9d464-c7c2-4980-ba4d-e3320373ed88\") "
Oct 30 22:05:37.154014 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:37.154000    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47d9d464-c7c2-4980-ba4d-e3320373ed88-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47d9d464-c7c2-4980-ba4d-e3320373ed88-workload-certs\") pod \"47d9d464-c7c2-4980-ba4d-e3320373ed88\" (UID: \"47d9d464-c7c2-4980-ba4d-e3320373ed88\") "
Oct 30 22:05:37.154407 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:37.154384    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47d9d464-c7c2-4980-ba4d-e3320373ed88-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47d9d464-c7c2-4980-ba4d-e3320373ed88-istio-envoy\") pod \"47d9d464-c7c2-4980-ba4d-e3320373ed88\" (UID: \"47d9d464-c7c2-4980-ba4d-e3320373ed88\") "
Oct 30 22:05:37.154526 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:37.154438    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/47d9d464-c7c2-4980-ba4d-e3320373ed88-kube-api-access-bspwc\" (UniqueName: \"kubernetes.io/projected/47d9d464-c7c2-4980-ba4d-e3320373ed88-kube-api-access-bspwc\") pod \"47d9d464-c7c2-4980-ba4d-e3320373ed88\" (UID: \"47d9d464-c7c2-4980-ba4d-e3320373ed88\") "
Oct 30 22:05:37.154600 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:37.153576    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47d9d464-c7c2-4980-ba4d-e3320373ed88-credential-socket" pod "47d9d464-c7c2-4980-ba4d-e3320373ed88" (UID: "47d9d464-c7c2-4980-ba4d-e3320373ed88"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:37.154693 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:37.153933    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47d9d464-c7c2-4980-ba4d-e3320373ed88-workload-socket" pod "47d9d464-c7c2-4980-ba4d-e3320373ed88" (UID: "47d9d464-c7c2-4980-ba4d-e3320373ed88"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:37.154782 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:37.154231    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47d9d464-c7c2-4980-ba4d-e3320373ed88-workload-certs" pod "47d9d464-c7c2-4980-ba4d-e3320373ed88" (UID: "47d9d464-c7c2-4980-ba4d-e3320373ed88"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:37.154874 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:37.154607    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47d9d464-c7c2-4980-ba4d-e3320373ed88-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47d9d464-c7c2-4980-ba4d-e3320373ed88-istio-data\") pod \"47d9d464-c7c2-4980-ba4d-e3320373ed88\" (UID: \"47d9d464-c7c2-4980-ba4d-e3320373ed88\") "
Oct 30 22:05:37.155027 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:37.155005    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/47d9d464-c7c2-4980-ba4d-e3320373ed88-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47d9d464-c7c2-4980-ba4d-e3320373ed88-istio-podinfo\") pod \"47d9d464-c7c2-4980-ba4d-e3320373ed88\" (UID: \"47d9d464-c7c2-4980-ba4d-e3320373ed88\") "
Oct 30 22:05:37.156940 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:37.155162    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47d9d464-c7c2-4980-ba4d-e3320373ed88-istio-data" pod "47d9d464-c7c2-4980-ba4d-e3320373ed88" (UID: "47d9d464-c7c2-4980-ba4d-e3320373ed88"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:37.157084 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:37.155281    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/47d9d464-c7c2-4980-ba4d-e3320373ed88-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47d9d464-c7c2-4980-ba4d-e3320373ed88-istiod-ca-cert\") pod \"47d9d464-c7c2-4980-ba4d-e3320373ed88\" (UID: \"47d9d464-c7c2-4980-ba4d-e3320373ed88\") "
Oct 30 22:05:37.157154 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:37.157118    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/47d9d464-c7c2-4980-ba4d-e3320373ed88-istio-token\" (UniqueName: \"kubernetes.io/projected/47d9d464-c7c2-4980-ba4d-e3320373ed88-istio-token\") pod \"47d9d464-c7c2-4980-ba4d-e3320373ed88\" (UID: \"47d9d464-c7c2-4980-ba4d-e3320373ed88\") "
Oct 30 22:05:37.157277 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:37.155658    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47d9d464-c7c2-4980-ba4d-e3320373ed88-istiod-ca-cert" pod "47d9d464-c7c2-4980-ba4d-e3320373ed88" (UID: "47d9d464-c7c2-4980-ba4d-e3320373ed88"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:05:37.157666 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:37.157640    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47d9d464-c7c2-4980-ba4d-e3320373ed88-istio-envoy" pod "47d9d464-c7c2-4980-ba4d-e3320373ed88" (UID: "47d9d464-c7c2-4980-ba4d-e3320373ed88"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:37.157947 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:37.157923    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47d9d464-c7c2-4980-ba4d-e3320373ed88-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:37.158066 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:37.157950    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47d9d464-c7c2-4980-ba4d-e3320373ed88-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:37.158066 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:37.157964    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47d9d464-c7c2-4980-ba4d-e3320373ed88-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:37.158066 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:37.157978    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47d9d464-c7c2-4980-ba4d-e3320373ed88-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:37.158066 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:37.157991    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47d9d464-c7c2-4980-ba4d-e3320373ed88-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:37.159258 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:37.159033    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47d9d464-c7c2-4980-ba4d-e3320373ed88-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:37.159258 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:37.159199    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/47d9d464-c7c2-4980-ba4d-e3320373ed88-istio-podinfo" pod "47d9d464-c7c2-4980-ba4d-e3320373ed88" (UID: "47d9d464-c7c2-4980-ba4d-e3320373ed88"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:05:37.173761 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:37.171714    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47d9d464-c7c2-4980-ba4d-e3320373ed88-istio-token" pod "47d9d464-c7c2-4980-ba4d-e3320373ed88" (UID: "47d9d464-c7c2-4980-ba4d-e3320373ed88"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:05:37.174206 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:37.174174    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47d9d464-c7c2-4980-ba4d-e3320373ed88-kube-api-access-bspwc" pod "47d9d464-c7c2-4980-ba4d-e3320373ed88" (UID: "47d9d464-c7c2-4980-ba4d-e3320373ed88"). InnerVolumeSpecName "kube-api-access-bspwc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:05:37.261031 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:37.260992    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bspwc\" (UniqueName: \"kubernetes.io/projected/47d9d464-c7c2-4980-ba4d-e3320373ed88-kube-api-access-bspwc\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:37.261031 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:37.261029    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47d9d464-c7c2-4980-ba4d-e3320373ed88-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:37.261597 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:37.261041    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47d9d464-c7c2-4980-ba4d-e3320373ed88-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:37.571060 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:37.570977    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"
Oct 30 22:05:37.571509 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:37.571486    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"
Oct 30 22:05:37.572337 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:37.572314    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"
Oct 30 22:05:37.897312 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:37.878763    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"
Oct 30 22:05:37.897312 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:37.880606    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"
Oct 30 22:05:37.917725 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:37.915320    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:37.918170 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:37.918148    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"
Oct 30 22:05:37.923286 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:37.922284    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"
Oct 30 22:05:37.933125 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:37.933091    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh" event={"ID":"47d9d464-c7c2-4980-ba4d-e3320373ed88","Type":"ContainerDied","Data":"36d126e8c3b1236bead43b361107f7a56b366e05d4ddb1072f4322ebf2fcba1c"}
Oct 30 22:05:38.133953 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:38.132277    4871 scope.go:122] "RemoveContainer" containerID="88cce796c963ab6a228d9846d6b5114217db53e246ee8de0a3c3871a35e3cfbc"
Oct 30 22:05:38.159650 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:38.158949    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5f3ddcc7-e832-42e9-97d4-1541f5a09044" path="/var/lib/kubelet/pods/5f3ddcc7-e832-42e9-97d4-1541f5a09044/volumes"
Oct 30 22:05:38.449357 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:38.449277    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"]
Oct 30 22:05:38.455572 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:38.455533    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-8c5vh"]
Oct 30 22:05:39.062416 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:39.062373    4871 scope.go:122] "RemoveContainer" containerID="74bafa3b50d112fcee4591e0ae3ec2df8e1b315731ffd4c74b4954ac58623da7"
Oct 30 22:05:39.428789 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:39.428752    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:39.614961 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:39.614902    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:39.828278 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:39.816629    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:40.175721 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:40.175653    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="47d9d464-c7c2-4980-ba4d-e3320373ed88" path="/var/lib/kubelet/pods/47d9d464-c7c2-4980-ba4d-e3320373ed88/volumes"
Oct 30 22:05:40.219179 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:40.219104    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-xtg4m"]
Oct 30 22:05:40.220181 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:40.220140    4871 cpu_manager.go:414] "containerMap: removing container" podUID="5f3ddcc7-e832-42e9-97d4-1541f5a09044" containerName="istio-proxy"
Oct 30 22:05:40.220181 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:40.220166    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5f3ddcc7-e832-42e9-97d4-1541f5a09044" containerName="istio-proxy"
Oct 30 22:05:40.220398 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:40.220188    4871 cpu_manager.go:414] "containerMap: removing container" podUID="78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd" containerName="istio-proxy"
Oct 30 22:05:40.220398 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:40.220196    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd" containerName="istio-proxy"
Oct 30 22:05:40.220398 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:40.220224    4871 cpu_manager.go:414] "containerMap: removing container" podUID="d9a09763-b456-469d-80c7-a787456003f4" containerName="istio-proxy"
Oct 30 22:05:40.220398 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:40.220249    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d9a09763-b456-469d-80c7-a787456003f4" containerName="istio-proxy"
Oct 30 22:05:40.220398 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:40.220265    4871 cpu_manager.go:414] "containerMap: removing container" podUID="84b9ee78-064c-4893-b81f-b9013b323448" containerName="istio-proxy"
Oct 30 22:05:40.220398 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:40.220271    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="84b9ee78-064c-4893-b81f-b9013b323448" containerName="istio-proxy"
Oct 30 22:05:40.220398 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:40.220289    4871 cpu_manager.go:414] "containerMap: removing container" podUID="33920f19-5306-4275-b275-3433be348592" containerName="istio-proxy"
Oct 30 22:05:40.220398 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:40.220297    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="33920f19-5306-4275-b275-3433be348592" containerName="istio-proxy"
Oct 30 22:05:40.220398 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:40.220312    4871 cpu_manager.go:414] "containerMap: removing container" podUID="47d9d464-c7c2-4980-ba4d-e3320373ed88" containerName="istio-proxy"
Oct 30 22:05:40.220398 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:40.220318    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="47d9d464-c7c2-4980-ba4d-e3320373ed88" containerName="istio-proxy"
Oct 30 22:05:40.220912 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:40.220496    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="78dcf4bb-0d68-4cd6-b280-0b8e070a4bcd" containerName="istio-proxy"
Oct 30 22:05:40.220912 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:40.220511    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="d9a09763-b456-469d-80c7-a787456003f4" containerName="istio-proxy"
Oct 30 22:05:40.220912 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:40.220519    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="84b9ee78-064c-4893-b81f-b9013b323448" containerName="istio-proxy"
Oct 30 22:05:40.220912 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:40.220530    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="33920f19-5306-4275-b275-3433be348592" containerName="istio-proxy"
Oct 30 22:05:40.220912 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:40.220540    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="47d9d464-c7c2-4980-ba4d-e3320373ed88" containerName="istio-proxy"
Oct 30 22:05:40.220912 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:40.220558    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="5f3ddcc7-e832-42e9-97d4-1541f5a09044" containerName="istio-proxy"
Oct 30 22:05:40.233204 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:40.233172    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-xtg4m"]
Oct 30 22:05:40.233715 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:40.233674    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-xtg4m"
Oct 30 22:05:40.270355 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:40.270319    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc280bff-3593-4ae0-b27d-88339a1e100b-plugins-config-volume\") pod \"payload-processing-e2e-mt-4c260505-68cf845c4f-xtg4m\" (UID: \"bc280bff-3593-4ae0-b27d-88339a1e100b\") " pod="openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-xtg4m"
Oct 30 22:05:40.270529 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:40.270481    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfwcn\" (UniqueName: \"kubernetes.io/projected/bc280bff-3593-4ae0-b27d-88339a1e100b-kube-api-access-zfwcn\") pod \"payload-processing-e2e-mt-4c260505-68cf845c4f-xtg4m\" (UID: \"bc280bff-3593-4ae0-b27d-88339a1e100b\") " pod="openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-xtg4m"
Oct 30 22:05:40.293216 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:40.293183    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-4c260505\""
Oct 30 22:05:40.323358 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:40.323319    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-4c260505-dockercfg-8n6k2\""
Oct 30 22:05:40.372052 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:40.372009    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc280bff-3593-4ae0-b27d-88339a1e100b-plugins-config-volume\") pod \"payload-processing-e2e-mt-4c260505-68cf845c4f-xtg4m\" (UID: \"bc280bff-3593-4ae0-b27d-88339a1e100b\") " pod="openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-xtg4m"
Oct 30 22:05:40.372290 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:40.372172    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfwcn\" (UniqueName: \"kubernetes.io/projected/bc280bff-3593-4ae0-b27d-88339a1e100b-kube-api-access-zfwcn\") pod \"payload-processing-e2e-mt-4c260505-68cf845c4f-xtg4m\" (UID: \"bc280bff-3593-4ae0-b27d-88339a1e100b\") " pod="openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-xtg4m"
Oct 30 22:05:40.372928 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:40.372902    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc280bff-3593-4ae0-b27d-88339a1e100b-plugins-config-volume\") pod \"payload-processing-e2e-mt-4c260505-68cf845c4f-xtg4m\" (UID: \"bc280bff-3593-4ae0-b27d-88339a1e100b\") " pod="openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-xtg4m"
Oct 30 22:05:40.430917 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:40.430842    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfwcn\" (UniqueName: \"kubernetes.io/projected/bc280bff-3593-4ae0-b27d-88339a1e100b-kube-api-access-zfwcn\") pod \"payload-processing-e2e-mt-4c260505-68cf845c4f-xtg4m\" (UID: \"bc280bff-3593-4ae0-b27d-88339a1e100b\") " pod="openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-xtg4m"
Oct 30 22:05:40.552833 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:40.552801    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-xtg4m"
Oct 30 22:05:40.941519 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:40.941094    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-xtg4m"]
Oct 30 22:05:41.236797 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:41.236698    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-xtg4m" event={"ID":"bc280bff-3593-4ae0-b27d-88339a1e100b","Type":"ContainerStarted","Data":"9809df9d7bc34e79c33e0ea13934e306fa27f9b65c2ad3d5b7d6d395e8b423ad"}
Oct 30 22:05:41.236797 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:41.236765    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-xtg4m" event={"ID":"bc280bff-3593-4ae0-b27d-88339a1e100b","Type":"ContainerStarted","Data":"f2e95bd8729302fdf1ebdced36ae7140d125ee229421abf5261a158c3f3d3234"}
Oct 30 22:05:41.236797 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:41.236796    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-xtg4m"
Oct 30 22:05:41.353268 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:41.353155    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-xtg4m" podStartSLOduration=2.353136558 podStartE2EDuration="2.353136558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:05:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:05:41.345954978 +0000 UTC m=+1683.523443594" watchObservedRunningTime="2026-10-30 22:05:41.353136558 +0000 UTC m=+1683.530625176"
Oct 30 22:05:44.925262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:44.921808    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:45.275411 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:45.275325    4871 generic.go:356] "Generic (PLEG): container finished" podID="e00cf9ef-6182-4fc8-b564-ea2425c9e5a5" containerID="389157db519b317d7fd8a6435373d87d78a5106c90d871b73e23738271a5ab58" exitCode=0
Oct 30 22:05:45.275411 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:45.275394    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp" event={"ID":"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5","Type":"ContainerDied","Data":"389157db519b317d7fd8a6435373d87d78a5106c90d871b73e23738271a5ab58"}
Oct 30 22:05:45.275862 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:45.275848    4871 scope.go:122] "RemoveContainer" containerID="389157db519b317d7fd8a6435373d87d78a5106c90d871b73e23738271a5ab58"
Oct 30 22:05:45.277522 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:45.277492    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:05:45.277603 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:45.277567    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:05:45.277603 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:45.277600    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:05:46.287302 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:46.287259    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp" event={"ID":"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5","Type":"ContainerStarted","Data":"02d412827d862380b10acbe699f6fcce68156210b81c32b1fa8c2a28c81e62db"}
Oct 30 22:05:47.035270 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:47.035221    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"
Oct 30 22:05:47.035507 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:47.035448    4871 patch_prober.go:29] interesting pod/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused" start-of-body=
Oct 30 22:05:47.035598 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:47.035500    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp" podUID="e00cf9ef-6182-4fc8-b564-ea2425c9e5a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused"
Oct 30 22:05:47.227695 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:47.227456    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:47.384646 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:47.384607    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:47.996312 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:47.996270    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:48.035564 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:48.035526    4871 patch_prober.go:29] interesting pod/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused" start-of-body=
Oct 30 22:05:48.035791 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:48.035606    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp" podUID="e00cf9ef-6182-4fc8-b564-ea2425c9e5a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused"
Oct 30 22:05:48.884355 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:48.884306    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"]
Oct 30 22:05:48.894910 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:48.890110    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"
Oct 30 22:05:48.894910 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:48.891430    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"]
Oct 30 22:05:48.942952 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:48.942912    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-2f6bc1-openshift-default-dockercfg-szqsw\""
Oct 30 22:05:48.943144 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:48.943100    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:49.029262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:48.998451    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhmlm\" (UniqueName: \"kubernetes.io/projected/42c86b8e-04b6-49fa-af33-9bc73297d162-kube-api-access-rhmlm\") pod \"e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh\" (UID: \"42c86b8e-04b6-49fa-af33-9bc73297d162\") " pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"
Oct 30 22:05:49.029262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:48.998557    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42c86b8e-04b6-49fa-af33-9bc73297d162-credential-socket\") pod \"e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh\" (UID: \"42c86b8e-04b6-49fa-af33-9bc73297d162\") " pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"
Oct 30 22:05:49.029262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:48.998590    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42c86b8e-04b6-49fa-af33-9bc73297d162-istio-data\") pod \"e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh\" (UID: \"42c86b8e-04b6-49fa-af33-9bc73297d162\") " pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"
Oct 30 22:05:49.029262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:48.998702    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42c86b8e-04b6-49fa-af33-9bc73297d162-workload-certs\") pod \"e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh\" (UID: \"42c86b8e-04b6-49fa-af33-9bc73297d162\") " pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"
Oct 30 22:05:49.029262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:48.998736    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42c86b8e-04b6-49fa-af33-9bc73297d162-istiod-ca-cert\") pod \"e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh\" (UID: \"42c86b8e-04b6-49fa-af33-9bc73297d162\") " pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"
Oct 30 22:05:49.029262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:48.998805    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42c86b8e-04b6-49fa-af33-9bc73297d162-istio-podinfo\") pod \"e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh\" (UID: \"42c86b8e-04b6-49fa-af33-9bc73297d162\") " pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"
Oct 30 22:05:49.029262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:48.998931    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42c86b8e-04b6-49fa-af33-9bc73297d162-workload-socket\") pod \"e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh\" (UID: \"42c86b8e-04b6-49fa-af33-9bc73297d162\") " pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"
Oct 30 22:05:49.029262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:48.998983    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42c86b8e-04b6-49fa-af33-9bc73297d162-istio-token\") pod \"e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh\" (UID: \"42c86b8e-04b6-49fa-af33-9bc73297d162\") " pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"
Oct 30 22:05:49.029262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:48.999111    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42c86b8e-04b6-49fa-af33-9bc73297d162-istio-envoy\") pod \"e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh\" (UID: \"42c86b8e-04b6-49fa-af33-9bc73297d162\") " pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"
Oct 30 22:05:49.057331 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:49.051440    4871 patch_prober.go:29] interesting pod/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused" start-of-body=
Oct 30 22:05:49.057331 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:49.051546    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp" podUID="e00cf9ef-6182-4fc8-b564-ea2425c9e5a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused"
Oct 30 22:05:49.103488 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:49.102015    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42c86b8e-04b6-49fa-af33-9bc73297d162-istio-envoy\") pod \"e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh\" (UID: \"42c86b8e-04b6-49fa-af33-9bc73297d162\") " pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"
Oct 30 22:05:49.103488 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:49.102084    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhmlm\" (UniqueName: \"kubernetes.io/projected/42c86b8e-04b6-49fa-af33-9bc73297d162-kube-api-access-rhmlm\") pod \"e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh\" (UID: \"42c86b8e-04b6-49fa-af33-9bc73297d162\") " pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"
Oct 30 22:05:49.103488 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:49.102150    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42c86b8e-04b6-49fa-af33-9bc73297d162-credential-socket\") pod \"e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh\" (UID: \"42c86b8e-04b6-49fa-af33-9bc73297d162\") " pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"
Oct 30 22:05:49.103488 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:49.102174    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42c86b8e-04b6-49fa-af33-9bc73297d162-istio-data\") pod \"e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh\" (UID: \"42c86b8e-04b6-49fa-af33-9bc73297d162\") " pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"
Oct 30 22:05:49.103488 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:49.102292    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42c86b8e-04b6-49fa-af33-9bc73297d162-workload-certs\") pod \"e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh\" (UID: \"42c86b8e-04b6-49fa-af33-9bc73297d162\") " pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"
Oct 30 22:05:49.103488 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:49.102325    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42c86b8e-04b6-49fa-af33-9bc73297d162-istiod-ca-cert\") pod \"e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh\" (UID: \"42c86b8e-04b6-49fa-af33-9bc73297d162\") " pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"
Oct 30 22:05:49.103488 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:49.102397    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42c86b8e-04b6-49fa-af33-9bc73297d162-istio-podinfo\") pod \"e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh\" (UID: \"42c86b8e-04b6-49fa-af33-9bc73297d162\") " pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"
Oct 30 22:05:49.103488 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:49.102521    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42c86b8e-04b6-49fa-af33-9bc73297d162-workload-socket\") pod \"e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh\" (UID: \"42c86b8e-04b6-49fa-af33-9bc73297d162\") " pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"
Oct 30 22:05:49.103488 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:49.102584    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42c86b8e-04b6-49fa-af33-9bc73297d162-istio-token\") pod \"e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh\" (UID: \"42c86b8e-04b6-49fa-af33-9bc73297d162\") " pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"
Oct 30 22:05:49.106170 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:49.104970    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42c86b8e-04b6-49fa-af33-9bc73297d162-workload-certs\") pod \"e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh\" (UID: \"42c86b8e-04b6-49fa-af33-9bc73297d162\") " pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"
Oct 30 22:05:49.106170 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:49.105517    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42c86b8e-04b6-49fa-af33-9bc73297d162-credential-socket\") pod \"e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh\" (UID: \"42c86b8e-04b6-49fa-af33-9bc73297d162\") " pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"
Oct 30 22:05:49.106170 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:49.105768    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42c86b8e-04b6-49fa-af33-9bc73297d162-istio-data\") pod \"e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh\" (UID: \"42c86b8e-04b6-49fa-af33-9bc73297d162\") " pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"
Oct 30 22:05:49.132579 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:49.117129    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42c86b8e-04b6-49fa-af33-9bc73297d162-workload-socket\") pod \"e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh\" (UID: \"42c86b8e-04b6-49fa-af33-9bc73297d162\") " pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"
Oct 30 22:05:49.148563 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:49.145760    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42c86b8e-04b6-49fa-af33-9bc73297d162-istiod-ca-cert\") pod \"e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh\" (UID: \"42c86b8e-04b6-49fa-af33-9bc73297d162\") " pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"
Oct 30 22:05:49.184029 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:49.183318    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:49.237886 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:49.237850    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42c86b8e-04b6-49fa-af33-9bc73297d162-istio-envoy\") pod \"e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh\" (UID: \"42c86b8e-04b6-49fa-af33-9bc73297d162\") " pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"
Oct 30 22:05:49.264315 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:49.264162    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42c86b8e-04b6-49fa-af33-9bc73297d162-istio-token\") pod \"e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh\" (UID: \"42c86b8e-04b6-49fa-af33-9bc73297d162\") " pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"
Oct 30 22:05:49.265214 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:49.265152    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42c86b8e-04b6-49fa-af33-9bc73297d162-istio-podinfo\") pod \"e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh\" (UID: \"42c86b8e-04b6-49fa-af33-9bc73297d162\") " pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"
Oct 30 22:05:49.268190 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:49.268141    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhmlm\" (UniqueName: \"kubernetes.io/projected/42c86b8e-04b6-49fa-af33-9bc73297d162-kube-api-access-rhmlm\") pod \"e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh\" (UID: \"42c86b8e-04b6-49fa-af33-9bc73297d162\") " pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"
Oct 30 22:05:49.550632 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:49.550532    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"
Oct 30 22:05:50.038265 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:50.036142    4871 patch_prober.go:29] interesting pod/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused" start-of-body=
Oct 30 22:05:50.038265 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:50.036207    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp" podUID="e00cf9ef-6182-4fc8-b564-ea2425c9e5a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused"
Oct 30 22:05:50.038870 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:50.038303    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"]
Oct 30 22:05:50.043364 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:50.043324    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:05:50.043619 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:50.043592    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:05:50.043752 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:50.043645    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:05:50.058309 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:50.058276    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:50.357302 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:50.357202    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh" event={"ID":"42c86b8e-04b6-49fa-af33-9bc73297d162","Type":"ContainerStarted","Data":"0f027d9405f9db17345e77c83cb859288cbc11f8f45e5c9c2349f0487b3bcb21"}
Oct 30 22:05:50.357302 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:50.357274    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh" event={"ID":"42c86b8e-04b6-49fa-af33-9bc73297d162","Type":"ContainerStarted","Data":"7509a503aab9cb9721649dc7e0726795af5af3ca8d1a46091b8b434a0f18551d"}
Oct 30 22:05:50.488551 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:50.488477    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh" podStartSLOduration=2.4884603739999998 podStartE2EDuration="2.488460374s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:05:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:05:50.483898038 +0000 UTC m=+1692.661386678" watchObservedRunningTime="2026-10-30 22:05:50.488460374 +0000 UTC m=+1692.665948991"
Oct 30 22:05:50.550975 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:50.550929    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"
Oct 30 22:05:51.036169 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:51.036132    4871 patch_prober.go:29] interesting pod/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused" start-of-body=
Oct 30 22:05:51.036435 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:51.036192    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp" podUID="e00cf9ef-6182-4fc8-b564-ea2425c9e5a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused"
Oct 30 22:05:51.557137 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:51.557100    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"
Oct 30 22:05:52.041860 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:52.041828    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"
Oct 30 22:05:52.042338 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:52.042319    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"
Oct 30 22:05:52.043095 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:52.043075    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"
Oct 30 22:05:52.249878 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:52.249848    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-xtg4m"
Oct 30 22:05:52.374532 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:52.374496    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"
Oct 30 22:05:52.375580 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:52.375557    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"
Oct 30 22:05:55.101787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:05:55.101704    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:00.209959 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:00.209926    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:02.929782 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:02.929742    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:03.119997 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:03.119948    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:03.286683 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:03.286598    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:03.519796 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:03.519745    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:05.450372 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:05.450305    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:05.650457 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:05.650406    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:05.698618 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:06:05.698578    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:06:10.182255 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:10.182209    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:10.404152 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:10.404103    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:10.556785 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:10.556691    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:14.976264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:14.976213    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4c260505-796c8fbff9-tdghh"]
Oct 30 22:06:14.980502 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:14.980481    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4c260505-796c8fbff9-tdghh"
Oct 30 22:06:15.063803 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:15.063763    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctg8k\" (UniqueName: \"kubernetes.io/projected/80d35579-62f6-47a6-bcab-37f16bec576a-kube-api-access-ctg8k\") pod \"maas-api-e2e-mt-4c260505-796c8fbff9-tdghh\" (UID: \"80d35579-62f6-47a6-bcab-37f16bec576a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4c260505-796c8fbff9-tdghh"
Oct 30 22:06:15.064017 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:15.063949    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80d35579-62f6-47a6-bcab-37f16bec576a-maas-api-tls\") pod \"maas-api-e2e-mt-4c260505-796c8fbff9-tdghh\" (UID: \"80d35579-62f6-47a6-bcab-37f16bec576a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4c260505-796c8fbff9-tdghh"
Oct 30 22:06:15.064274 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:15.064215    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80d35579-62f6-47a6-bcab-37f16bec576a-metrics-certs\") pod \"maas-api-e2e-mt-4c260505-796c8fbff9-tdghh\" (UID: \"80d35579-62f6-47a6-bcab-37f16bec576a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4c260505-796c8fbff9-tdghh"
Oct 30 22:06:15.064496 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:15.064476    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80d35579-62f6-47a6-bcab-37f16bec576a-openshift-service-ca\") pod \"maas-api-e2e-mt-4c260505-796c8fbff9-tdghh\" (UID: \"80d35579-62f6-47a6-bcab-37f16bec576a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4c260505-796c8fbff9-tdghh"
Oct 30 22:06:15.078896 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:15.078862    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-4c260505-serving-cert\""
Oct 30 22:06:15.084684 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:15.084652    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4c260505-796c8fbff9-tdghh"]
Oct 30 22:06:15.165592 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:15.165557    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:15.165780 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:15.165673    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80d35579-62f6-47a6-bcab-37f16bec576a-metrics-certs\") pod \"maas-api-e2e-mt-4c260505-796c8fbff9-tdghh\" (UID: \"80d35579-62f6-47a6-bcab-37f16bec576a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4c260505-796c8fbff9-tdghh"
Oct 30 22:06:15.165780 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:15.165714    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80d35579-62f6-47a6-bcab-37f16bec576a-openshift-service-ca\") pod \"maas-api-e2e-mt-4c260505-796c8fbff9-tdghh\" (UID: \"80d35579-62f6-47a6-bcab-37f16bec576a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4c260505-796c8fbff9-tdghh"
Oct 30 22:06:15.165907 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:15.165874    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctg8k\" (UniqueName: \"kubernetes.io/projected/80d35579-62f6-47a6-bcab-37f16bec576a-kube-api-access-ctg8k\") pod \"maas-api-e2e-mt-4c260505-796c8fbff9-tdghh\" (UID: \"80d35579-62f6-47a6-bcab-37f16bec576a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4c260505-796c8fbff9-tdghh"
Oct 30 22:06:15.165965 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:15.165939    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80d35579-62f6-47a6-bcab-37f16bec576a-maas-api-tls\") pod \"maas-api-e2e-mt-4c260505-796c8fbff9-tdghh\" (UID: \"80d35579-62f6-47a6-bcab-37f16bec576a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4c260505-796c8fbff9-tdghh"
Oct 30 22:06:15.166128 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:06:15.166107    4871 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-4c260505-serving-cert: secret "maas-api-e2e-mt-4c260505-serving-cert" not found
Oct 30 22:06:15.166205 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:06:15.166183    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80d35579-62f6-47a6-bcab-37f16bec576a-maas-api-tls podName:80d35579-62f6-47a6-bcab-37f16bec576a nodeName:}" failed. No retries permitted until 2026-10-30 22:06:15.66615807 +0000 UTC m=+1717.843646665 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/80d35579-62f6-47a6-bcab-37f16bec576a-maas-api-tls") pod "maas-api-e2e-mt-4c260505-796c8fbff9-tdghh" (UID: "80d35579-62f6-47a6-bcab-37f16bec576a") : secret "maas-api-e2e-mt-4c260505-serving-cert" not found
Oct 30 22:06:15.166511 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:15.166490    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80d35579-62f6-47a6-bcab-37f16bec576a-openshift-service-ca\") pod \"maas-api-e2e-mt-4c260505-796c8fbff9-tdghh\" (UID: \"80d35579-62f6-47a6-bcab-37f16bec576a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4c260505-796c8fbff9-tdghh"
Oct 30 22:06:15.168361 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:15.168340    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80d35579-62f6-47a6-bcab-37f16bec576a-metrics-certs\") pod \"maas-api-e2e-mt-4c260505-796c8fbff9-tdghh\" (UID: \"80d35579-62f6-47a6-bcab-37f16bec576a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4c260505-796c8fbff9-tdghh"
Oct 30 22:06:15.268894 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:15.267411    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctg8k\" (UniqueName: \"kubernetes.io/projected/80d35579-62f6-47a6-bcab-37f16bec576a-kube-api-access-ctg8k\") pod \"maas-api-e2e-mt-4c260505-796c8fbff9-tdghh\" (UID: \"80d35579-62f6-47a6-bcab-37f16bec576a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4c260505-796c8fbff9-tdghh"
Oct 30 22:06:15.338623 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:15.338588    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:15.537436 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:15.537225    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:15.676416 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:15.676377    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80d35579-62f6-47a6-bcab-37f16bec576a-maas-api-tls\") pod \"maas-api-e2e-mt-4c260505-796c8fbff9-tdghh\" (UID: \"80d35579-62f6-47a6-bcab-37f16bec576a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4c260505-796c8fbff9-tdghh"
Oct 30 22:06:15.678856 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:15.678832    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80d35579-62f6-47a6-bcab-37f16bec576a-maas-api-tls\") pod \"maas-api-e2e-mt-4c260505-796c8fbff9-tdghh\" (UID: \"80d35579-62f6-47a6-bcab-37f16bec576a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4c260505-796c8fbff9-tdghh"
Oct 30 22:06:15.895862 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:15.895829    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4c260505-796c8fbff9-tdghh"
Oct 30 22:06:16.279067 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:06:16.279034    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80d35579_62f6_47a6_bcab_37f16bec576a.slice/crio-63dc017c9a0f9dd911f93f118ef5c955e367730937db06a3175234d07ad078f0 WatchSource:0}: Error finding container 63dc017c9a0f9dd911f93f118ef5c955e367730937db06a3175234d07ad078f0: Status 404 returned error can't find the container with id 63dc017c9a0f9dd911f93f118ef5c955e367730937db06a3175234d07ad078f0
Oct 30 22:06:16.280074 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:16.280037    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4c260505-796c8fbff9-tdghh"]
Oct 30 22:06:16.960996 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:16.960957    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4c260505-796c8fbff9-tdghh" event={"ID":"80d35579-62f6-47a6-bcab-37f16bec576a","Type":"ContainerStarted","Data":"63dc017c9a0f9dd911f93f118ef5c955e367730937db06a3175234d07ad078f0"}
Oct 30 22:06:18.017262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:18.016894    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4c260505-796c8fbff9-tdghh"
Oct 30 22:06:18.017262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:18.016960    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4c260505-796c8fbff9-tdghh" event={"ID":"80d35579-62f6-47a6-bcab-37f16bec576a","Type":"ContainerStarted","Data":"ed934a3670d468e7b62c3e587187b2ed25a72b0b8d8d57519a18ab4e93addd80"}
Oct 30 22:06:18.129412 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:18.129352    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4c260505-796c8fbff9-tdghh" podStartSLOduration=3.228219513 podStartE2EDuration="4.129339285s" totalImagesPullingTime="901.119772ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:06:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:06:18.126267425 +0000 UTC m=+1720.303756036" watchObservedRunningTime="2026-10-30 22:06:18.129339285 +0000 UTC m=+1720.306827902"
Oct 30 22:06:19.005042 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:19.004999    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:19.170623 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:19.170568    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:19.784755 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:19.784720    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-97f097-85dddff48-dj9xs"]
Oct 30 22:06:19.791008 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:19.790763    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-97f097-85dddff48-dj9xs"
Oct 30 22:06:19.845730 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:19.845695    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-97f097-85dddff48-dj9xs"]
Oct 30 22:06:19.852033 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:19.851997    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95aa3a5c-4534-4ca7-a4ca-f3db06d36633-metrics-certs\") pod \"maas-api-e2e-api-b-97f097-85dddff48-dj9xs\" (UID: \"95aa3a5c-4534-4ca7-a4ca-f3db06d36633\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-97f097-85dddff48-dj9xs"
Oct 30 22:06:19.852285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:19.852203    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95aa3a5c-4534-4ca7-a4ca-f3db06d36633-openshift-service-ca\") pod \"maas-api-e2e-api-b-97f097-85dddff48-dj9xs\" (UID: \"95aa3a5c-4534-4ca7-a4ca-f3db06d36633\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-97f097-85dddff48-dj9xs"
Oct 30 22:06:19.852427 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:19.852376    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ls64x\" (UniqueName: \"kubernetes.io/projected/95aa3a5c-4534-4ca7-a4ca-f3db06d36633-kube-api-access-ls64x\") pod \"maas-api-e2e-api-b-97f097-85dddff48-dj9xs\" (UID: \"95aa3a5c-4534-4ca7-a4ca-f3db06d36633\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-97f097-85dddff48-dj9xs"
Oct 30 22:06:19.852513 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:19.852450    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95aa3a5c-4534-4ca7-a4ca-f3db06d36633-maas-api-tls\") pod \"maas-api-e2e-api-b-97f097-85dddff48-dj9xs\" (UID: \"95aa3a5c-4534-4ca7-a4ca-f3db06d36633\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-97f097-85dddff48-dj9xs"
Oct 30 22:06:19.887350 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:19.887314    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-97f097-serving-cert\""
Oct 30 22:06:19.954212 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:19.954173    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95aa3a5c-4534-4ca7-a4ca-f3db06d36633-openshift-service-ca\") pod \"maas-api-e2e-api-b-97f097-85dddff48-dj9xs\" (UID: \"95aa3a5c-4534-4ca7-a4ca-f3db06d36633\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-97f097-85dddff48-dj9xs"
Oct 30 22:06:19.954402 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:19.954291    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ls64x\" (UniqueName: \"kubernetes.io/projected/95aa3a5c-4534-4ca7-a4ca-f3db06d36633-kube-api-access-ls64x\") pod \"maas-api-e2e-api-b-97f097-85dddff48-dj9xs\" (UID: \"95aa3a5c-4534-4ca7-a4ca-f3db06d36633\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-97f097-85dddff48-dj9xs"
Oct 30 22:06:19.954402 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:19.954344    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95aa3a5c-4534-4ca7-a4ca-f3db06d36633-maas-api-tls\") pod \"maas-api-e2e-api-b-97f097-85dddff48-dj9xs\" (UID: \"95aa3a5c-4534-4ca7-a4ca-f3db06d36633\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-97f097-85dddff48-dj9xs"
Oct 30 22:06:19.954498 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:19.954440    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95aa3a5c-4534-4ca7-a4ca-f3db06d36633-metrics-certs\") pod \"maas-api-e2e-api-b-97f097-85dddff48-dj9xs\" (UID: \"95aa3a5c-4534-4ca7-a4ca-f3db06d36633\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-97f097-85dddff48-dj9xs"
Oct 30 22:06:19.954599 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:06:19.954569    4871 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-97f097-serving-cert: secret "maas-api-e2e-api-b-97f097-serving-cert" not found
Oct 30 22:06:19.954798 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:06:19.954657    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95aa3a5c-4534-4ca7-a4ca-f3db06d36633-maas-api-tls podName:95aa3a5c-4534-4ca7-a4ca-f3db06d36633 nodeName:}" failed. No retries permitted until 2026-10-30 22:06:20.454637098 +0000 UTC m=+1722.632125710 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/95aa3a5c-4534-4ca7-a4ca-f3db06d36633-maas-api-tls") pod "maas-api-e2e-api-b-97f097-85dddff48-dj9xs" (UID: "95aa3a5c-4534-4ca7-a4ca-f3db06d36633") : secret "maas-api-e2e-api-b-97f097-serving-cert" not found
Oct 30 22:06:19.955610 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:19.955582    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95aa3a5c-4534-4ca7-a4ca-f3db06d36633-openshift-service-ca\") pod \"maas-api-e2e-api-b-97f097-85dddff48-dj9xs\" (UID: \"95aa3a5c-4534-4ca7-a4ca-f3db06d36633\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-97f097-85dddff48-dj9xs"
Oct 30 22:06:19.957206 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:19.957187    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95aa3a5c-4534-4ca7-a4ca-f3db06d36633-metrics-certs\") pod \"maas-api-e2e-api-b-97f097-85dddff48-dj9xs\" (UID: \"95aa3a5c-4534-4ca7-a4ca-f3db06d36633\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-97f097-85dddff48-dj9xs"
Oct 30 22:06:19.975128 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:19.975100    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:20.013541 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:20.013510    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ls64x\" (UniqueName: \"kubernetes.io/projected/95aa3a5c-4534-4ca7-a4ca-f3db06d36633-kube-api-access-ls64x\") pod \"maas-api-e2e-api-b-97f097-85dddff48-dj9xs\" (UID: \"95aa3a5c-4534-4ca7-a4ca-f3db06d36633\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-97f097-85dddff48-dj9xs"
Oct 30 22:06:20.121983 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:20.121934    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:20.294260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:20.284818    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:20.470308 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:20.469408    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95aa3a5c-4534-4ca7-a4ca-f3db06d36633-maas-api-tls\") pod \"maas-api-e2e-api-b-97f097-85dddff48-dj9xs\" (UID: \"95aa3a5c-4534-4ca7-a4ca-f3db06d36633\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-97f097-85dddff48-dj9xs"
Oct 30 22:06:20.508257 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:20.507592    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95aa3a5c-4534-4ca7-a4ca-f3db06d36633-maas-api-tls\") pod \"maas-api-e2e-api-b-97f097-85dddff48-dj9xs\" (UID: \"95aa3a5c-4534-4ca7-a4ca-f3db06d36633\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-97f097-85dddff48-dj9xs"
Oct 30 22:06:20.711585 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:20.711536    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-97f097-85dddff48-dj9xs"
Oct 30 22:06:21.168079 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:06:21.168040    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95aa3a5c_4534_4ca7_a4ca_f3db06d36633.slice/crio-9305f23ba13bfd0ff9c51dc0ec59b1ad4ecb99b98bd7c96c66861944cd5fddbe WatchSource:0}: Error finding container 9305f23ba13bfd0ff9c51dc0ec59b1ad4ecb99b98bd7c96c66861944cd5fddbe: Status 404 returned error can't find the container with id 9305f23ba13bfd0ff9c51dc0ec59b1ad4ecb99b98bd7c96c66861944cd5fddbe
Oct 30 22:06:21.169332 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:21.169305    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-97f097-85dddff48-dj9xs"]
Oct 30 22:06:22.048317 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:22.048277    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-97f097-85dddff48-dj9xs" event={"ID":"95aa3a5c-4534-4ca7-a4ca-f3db06d36633","Type":"ContainerStarted","Data":"9305f23ba13bfd0ff9c51dc0ec59b1ad4ecb99b98bd7c96c66861944cd5fddbe"}
Oct 30 22:06:23.057794 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:23.057758    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-97f097-85dddff48-dj9xs" event={"ID":"95aa3a5c-4534-4ca7-a4ca-f3db06d36633","Type":"ContainerStarted","Data":"496f0bb133b41c2c10cbdebd5fa4f5a97c12970bd8d369ffb638e13feed1858e"}
Oct 30 22:06:23.058203 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:23.057826    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-97f097-85dddff48-dj9xs"
Oct 30 22:06:23.215003 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:23.214935    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-97f097-85dddff48-dj9xs" podStartSLOduration=3.365766037 podStartE2EDuration="4.214916869s" totalImagesPullingTime="849.150832ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:06:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:06:23.210982139 +0000 UTC m=+1725.388470774" watchObservedRunningTime="2026-10-30 22:06:23.214916869 +0000 UTC m=+1725.392405529"
Oct 30 22:06:24.027103 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:24.027073    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4c260505-796c8fbff9-tdghh"
Oct 30 22:06:24.909507 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:24.909463    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:26.409748 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:26.409710    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:27.309029 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:27.308990    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:29.071604 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:29.071574    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-97f097-85dddff48-dj9xs"
Oct 30 22:06:29.503817 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:29.503780    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:34.213924 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:34.213883    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:35.570412 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:06:35.570380    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:06:38.261717 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:38.261674    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:38.898733 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:38.898697    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:43.517503 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:43.517461    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:48.012149 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:48.012066    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:52.597572 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:52.597530    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:57.109843 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:06:57.109808    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:01.379937 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:01.379898    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t"]
Oct 30 22:07:01.384448 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:01.384425    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t"
Oct 30 22:07:01.434836 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:01.434796    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-2f6bc1-serving-cert\""
Oct 30 22:07:01.482143 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:01.482105    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07f3e008-0adf-4515-b983-fba8739d5ad9-metrics-certs\") pod \"maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t\" (UID: \"07f3e008-0adf-4515-b983-fba8739d5ad9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t"
Oct 30 22:07:01.482392 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:01.482164    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07f3e008-0adf-4515-b983-fba8739d5ad9-maas-api-tls\") pod \"maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t\" (UID: \"07f3e008-0adf-4515-b983-fba8739d5ad9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t"
Oct 30 22:07:01.482475 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:01.482396    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07f3e008-0adf-4515-b983-fba8739d5ad9-openshift-service-ca\") pod \"maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t\" (UID: \"07f3e008-0adf-4515-b983-fba8739d5ad9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t"
Oct 30 22:07:01.482582 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:01.482560    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h27k4\" (UniqueName: \"kubernetes.io/projected/07f3e008-0adf-4515-b983-fba8739d5ad9-kube-api-access-h27k4\") pod \"maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t\" (UID: \"07f3e008-0adf-4515-b983-fba8739d5ad9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t"
Oct 30 22:07:01.487102 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:01.487076    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t"]
Oct 30 22:07:01.584196 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:01.584155    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07f3e008-0adf-4515-b983-fba8739d5ad9-metrics-certs\") pod \"maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t\" (UID: \"07f3e008-0adf-4515-b983-fba8739d5ad9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t"
Oct 30 22:07:01.584196 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:01.584202    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07f3e008-0adf-4515-b983-fba8739d5ad9-maas-api-tls\") pod \"maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t\" (UID: \"07f3e008-0adf-4515-b983-fba8739d5ad9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t"
Oct 30 22:07:01.584449 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:01.584312    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07f3e008-0adf-4515-b983-fba8739d5ad9-openshift-service-ca\") pod \"maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t\" (UID: \"07f3e008-0adf-4515-b983-fba8739d5ad9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t"
Oct 30 22:07:01.584488 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:01.584454    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h27k4\" (UniqueName: \"kubernetes.io/projected/07f3e008-0adf-4515-b983-fba8739d5ad9-kube-api-access-h27k4\") pod \"maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t\" (UID: \"07f3e008-0adf-4515-b983-fba8739d5ad9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t"
Oct 30 22:07:01.585065 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:01.585043    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07f3e008-0adf-4515-b983-fba8739d5ad9-openshift-service-ca\") pod \"maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t\" (UID: \"07f3e008-0adf-4515-b983-fba8739d5ad9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t"
Oct 30 22:07:01.586763 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:01.586736    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07f3e008-0adf-4515-b983-fba8739d5ad9-metrics-certs\") pod \"maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t\" (UID: \"07f3e008-0adf-4515-b983-fba8739d5ad9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t"
Oct 30 22:07:01.586857 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:01.586784    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07f3e008-0adf-4515-b983-fba8739d5ad9-maas-api-tls\") pod \"maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t\" (UID: \"07f3e008-0adf-4515-b983-fba8739d5ad9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t"
Oct 30 22:07:01.652166 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:01.652094    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h27k4\" (UniqueName: \"kubernetes.io/projected/07f3e008-0adf-4515-b983-fba8739d5ad9-kube-api-access-h27k4\") pod \"maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t\" (UID: \"07f3e008-0adf-4515-b983-fba8739d5ad9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t"
Oct 30 22:07:01.655084 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:01.655053    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:01.699521 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:01.699489    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t"
Oct 30 22:07:02.205836 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:02.205804    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t"]
Oct 30 22:07:02.206497 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:07:02.206470    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07f3e008_0adf_4515_b983_fba8739d5ad9.slice/crio-4f645bcddfdd11325f33d219bbef6d50aaf789e1dd085b951f1e936a8062aba6 WatchSource:0}: Error finding container 4f645bcddfdd11325f33d219bbef6d50aaf789e1dd085b951f1e936a8062aba6: Status 404 returned error can't find the container with id 4f645bcddfdd11325f33d219bbef6d50aaf789e1dd085b951f1e936a8062aba6
Oct 30 22:07:02.452364 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:02.452152    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t" event={"ID":"07f3e008-0adf-4515-b983-fba8739d5ad9","Type":"ContainerStarted","Data":"4f645bcddfdd11325f33d219bbef6d50aaf789e1dd085b951f1e936a8062aba6"}
Oct 30 22:07:03.030116 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:03.030072    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:03.484201 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:03.484154    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t" event={"ID":"07f3e008-0adf-4515-b983-fba8739d5ad9","Type":"ContainerStarted","Data":"fada876b7a5b74ebadd4146050e517cfcc16d9e14d2c2b56f9fbf004112ad41c"}
Oct 30 22:07:03.484738 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:03.484281    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t"
Oct 30 22:07:03.613692 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:03.613634    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t" podStartSLOduration=1.508482673 podStartE2EDuration="2.613618289s" totalImagesPullingTime="1.105135616s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:07:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:03.612454737 +0000 UTC m=+1765.789943354" watchObservedRunningTime="2026-10-30 22:07:03.613618289 +0000 UTC m=+1765.791106906"
Oct 30 22:07:05.572210 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:07:05.572164    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:07:06.333028 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:06.332985    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:06.484645 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:06.484595    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:06.987295 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:06.987257    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"]
Oct 30 22:07:06.992062 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:06.992039    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"
Oct 30 22:07:07.037712 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.037670    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"]
Oct 30 22:07:07.044752 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.044722    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"
Oct 30 22:07:07.081872 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.081833    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"]
Oct 30 22:07:07.090285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.087554    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a83867ab-e545-4023-8923-fc0124c9c638-istiod-ca-cert\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-wcxns\" (UID: \"a83867ab-e545-4023-8923-fc0124c9c638\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"
Oct 30 22:07:07.090285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.087635    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/957af2a6-bd01-49ed-a36a-1844daeebf2b-workload-certs\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-xdbqw\" (UID: \"957af2a6-bd01-49ed-a36a-1844daeebf2b\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"
Oct 30 22:07:07.090285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.087663    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a83867ab-e545-4023-8923-fc0124c9c638-istio-data\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-wcxns\" (UID: \"a83867ab-e545-4023-8923-fc0124c9c638\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"
Oct 30 22:07:07.090285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.087689    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/957af2a6-bd01-49ed-a36a-1844daeebf2b-istio-data\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-xdbqw\" (UID: \"957af2a6-bd01-49ed-a36a-1844daeebf2b\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"
Oct 30 22:07:07.090285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.087724    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/957af2a6-bd01-49ed-a36a-1844daeebf2b-workload-socket\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-xdbqw\" (UID: \"957af2a6-bd01-49ed-a36a-1844daeebf2b\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"
Oct 30 22:07:07.090285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.087785    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a83867ab-e545-4023-8923-fc0124c9c638-istio-podinfo\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-wcxns\" (UID: \"a83867ab-e545-4023-8923-fc0124c9c638\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"
Oct 30 22:07:07.090285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.087820    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/957af2a6-bd01-49ed-a36a-1844daeebf2b-istio-token\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-xdbqw\" (UID: \"957af2a6-bd01-49ed-a36a-1844daeebf2b\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"
Oct 30 22:07:07.090285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.087849    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbc99\" (UniqueName: \"kubernetes.io/projected/957af2a6-bd01-49ed-a36a-1844daeebf2b-kube-api-access-sbc99\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-xdbqw\" (UID: \"957af2a6-bd01-49ed-a36a-1844daeebf2b\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"
Oct 30 22:07:07.090285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.087883    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a83867ab-e545-4023-8923-fc0124c9c638-istio-token\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-wcxns\" (UID: \"a83867ab-e545-4023-8923-fc0124c9c638\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"
Oct 30 22:07:07.090285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.087983    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a83867ab-e545-4023-8923-fc0124c9c638-credential-socket\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-wcxns\" (UID: \"a83867ab-e545-4023-8923-fc0124c9c638\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"
Oct 30 22:07:07.090285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.088007    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/957af2a6-bd01-49ed-a36a-1844daeebf2b-istio-podinfo\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-xdbqw\" (UID: \"957af2a6-bd01-49ed-a36a-1844daeebf2b\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"
Oct 30 22:07:07.090285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.088063    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/957af2a6-bd01-49ed-a36a-1844daeebf2b-istiod-ca-cert\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-xdbqw\" (UID: \"957af2a6-bd01-49ed-a36a-1844daeebf2b\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"
Oct 30 22:07:07.090285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.088097    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a83867ab-e545-4023-8923-fc0124c9c638-workload-socket\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-wcxns\" (UID: \"a83867ab-e545-4023-8923-fc0124c9c638\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"
Oct 30 22:07:07.090285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.088142    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a83867ab-e545-4023-8923-fc0124c9c638-istio-envoy\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-wcxns\" (UID: \"a83867ab-e545-4023-8923-fc0124c9c638\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"
Oct 30 22:07:07.090285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.088253    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crfbm\" (UniqueName: \"kubernetes.io/projected/a83867ab-e545-4023-8923-fc0124c9c638-kube-api-access-crfbm\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-wcxns\" (UID: \"a83867ab-e545-4023-8923-fc0124c9c638\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"
Oct 30 22:07:07.090285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.088334    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a83867ab-e545-4023-8923-fc0124c9c638-workload-certs\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-wcxns\" (UID: \"a83867ab-e545-4023-8923-fc0124c9c638\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"
Oct 30 22:07:07.090285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.088386    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/957af2a6-bd01-49ed-a36a-1844daeebf2b-credential-socket\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-xdbqw\" (UID: \"957af2a6-bd01-49ed-a36a-1844daeebf2b\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"
Oct 30 22:07:07.090285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.088409    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/957af2a6-bd01-49ed-a36a-1844daeebf2b-istio-envoy\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-xdbqw\" (UID: \"957af2a6-bd01-49ed-a36a-1844daeebf2b\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"
Oct 30 22:07:07.100583 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.099167    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"]
Oct 30 22:07:07.190135 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.190097    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a83867ab-e545-4023-8923-fc0124c9c638-credential-socket\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-wcxns\" (UID: \"a83867ab-e545-4023-8923-fc0124c9c638\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"
Oct 30 22:07:07.190135 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.190137    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/957af2a6-bd01-49ed-a36a-1844daeebf2b-istio-podinfo\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-xdbqw\" (UID: \"957af2a6-bd01-49ed-a36a-1844daeebf2b\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"
Oct 30 22:07:07.190458 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.190283    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/957af2a6-bd01-49ed-a36a-1844daeebf2b-istiod-ca-cert\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-xdbqw\" (UID: \"957af2a6-bd01-49ed-a36a-1844daeebf2b\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"
Oct 30 22:07:07.190458 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.190339    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a83867ab-e545-4023-8923-fc0124c9c638-workload-socket\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-wcxns\" (UID: \"a83867ab-e545-4023-8923-fc0124c9c638\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"
Oct 30 22:07:07.190458 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.190385    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a83867ab-e545-4023-8923-fc0124c9c638-istio-envoy\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-wcxns\" (UID: \"a83867ab-e545-4023-8923-fc0124c9c638\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"
Oct 30 22:07:07.190635 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.190474    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crfbm\" (UniqueName: \"kubernetes.io/projected/a83867ab-e545-4023-8923-fc0124c9c638-kube-api-access-crfbm\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-wcxns\" (UID: \"a83867ab-e545-4023-8923-fc0124c9c638\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"
Oct 30 22:07:07.190635 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.190550    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a83867ab-e545-4023-8923-fc0124c9c638-workload-certs\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-wcxns\" (UID: \"a83867ab-e545-4023-8923-fc0124c9c638\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"
Oct 30 22:07:07.190635 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.190601    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/957af2a6-bd01-49ed-a36a-1844daeebf2b-credential-socket\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-xdbqw\" (UID: \"957af2a6-bd01-49ed-a36a-1844daeebf2b\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"
Oct 30 22:07:07.190635 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.190626    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/957af2a6-bd01-49ed-a36a-1844daeebf2b-istio-envoy\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-xdbqw\" (UID: \"957af2a6-bd01-49ed-a36a-1844daeebf2b\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"
Oct 30 22:07:07.190843 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.190664    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a83867ab-e545-4023-8923-fc0124c9c638-credential-socket\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-wcxns\" (UID: \"a83867ab-e545-4023-8923-fc0124c9c638\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"
Oct 30 22:07:07.190843 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.190683    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a83867ab-e545-4023-8923-fc0124c9c638-istiod-ca-cert\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-wcxns\" (UID: \"a83867ab-e545-4023-8923-fc0124c9c638\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"
Oct 30 22:07:07.190843 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.190714    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a83867ab-e545-4023-8923-fc0124c9c638-workload-socket\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-wcxns\" (UID: \"a83867ab-e545-4023-8923-fc0124c9c638\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"
Oct 30 22:07:07.190843 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.190754    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/957af2a6-bd01-49ed-a36a-1844daeebf2b-workload-certs\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-xdbqw\" (UID: \"957af2a6-bd01-49ed-a36a-1844daeebf2b\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"
Oct 30 22:07:07.190843 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.190787    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a83867ab-e545-4023-8923-fc0124c9c638-istio-data\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-wcxns\" (UID: \"a83867ab-e545-4023-8923-fc0124c9c638\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"
Oct 30 22:07:07.190843 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.190813    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/957af2a6-bd01-49ed-a36a-1844daeebf2b-istio-data\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-xdbqw\" (UID: \"957af2a6-bd01-49ed-a36a-1844daeebf2b\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"
Oct 30 22:07:07.191163 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.190847    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/957af2a6-bd01-49ed-a36a-1844daeebf2b-workload-socket\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-xdbqw\" (UID: \"957af2a6-bd01-49ed-a36a-1844daeebf2b\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"
Oct 30 22:07:07.191163 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.190903    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a83867ab-e545-4023-8923-fc0124c9c638-istio-podinfo\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-wcxns\" (UID: \"a83867ab-e545-4023-8923-fc0124c9c638\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"
Oct 30 22:07:07.191163 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.190938    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/957af2a6-bd01-49ed-a36a-1844daeebf2b-istio-token\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-xdbqw\" (UID: \"957af2a6-bd01-49ed-a36a-1844daeebf2b\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"
Oct 30 22:07:07.191163 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.190968    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbc99\" (UniqueName: \"kubernetes.io/projected/957af2a6-bd01-49ed-a36a-1844daeebf2b-kube-api-access-sbc99\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-xdbqw\" (UID: \"957af2a6-bd01-49ed-a36a-1844daeebf2b\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"
Oct 30 22:07:07.191163 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.191003    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a83867ab-e545-4023-8923-fc0124c9c638-istio-token\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-wcxns\" (UID: \"a83867ab-e545-4023-8923-fc0124c9c638\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"
Oct 30 22:07:07.191163 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.191028    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/957af2a6-bd01-49ed-a36a-1844daeebf2b-credential-socket\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-xdbqw\" (UID: \"957af2a6-bd01-49ed-a36a-1844daeebf2b\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"
Oct 30 22:07:07.191163 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.191147    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/957af2a6-bd01-49ed-a36a-1844daeebf2b-istiod-ca-cert\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-xdbqw\" (UID: \"957af2a6-bd01-49ed-a36a-1844daeebf2b\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"
Oct 30 22:07:07.191563 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.191330    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a83867ab-e545-4023-8923-fc0124c9c638-workload-certs\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-wcxns\" (UID: \"a83867ab-e545-4023-8923-fc0124c9c638\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"
Oct 30 22:07:07.191563 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.191451    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/957af2a6-bd01-49ed-a36a-1844daeebf2b-workload-socket\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-xdbqw\" (UID: \"957af2a6-bd01-49ed-a36a-1844daeebf2b\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"
Oct 30 22:07:07.191563 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.191477    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a83867ab-e545-4023-8923-fc0124c9c638-istiod-ca-cert\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-wcxns\" (UID: \"a83867ab-e545-4023-8923-fc0124c9c638\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"
Oct 30 22:07:07.191723 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.191686    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/957af2a6-bd01-49ed-a36a-1844daeebf2b-workload-certs\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-xdbqw\" (UID: \"957af2a6-bd01-49ed-a36a-1844daeebf2b\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"
Oct 30 22:07:07.192074 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.191888    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a83867ab-e545-4023-8923-fc0124c9c638-istio-data\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-wcxns\" (UID: \"a83867ab-e545-4023-8923-fc0124c9c638\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"
Oct 30 22:07:07.192074 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.192040    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/957af2a6-bd01-49ed-a36a-1844daeebf2b-istio-data\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-xdbqw\" (UID: \"957af2a6-bd01-49ed-a36a-1844daeebf2b\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"
Oct 30 22:07:07.193536 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.193511    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/957af2a6-bd01-49ed-a36a-1844daeebf2b-istio-podinfo\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-xdbqw\" (UID: \"957af2a6-bd01-49ed-a36a-1844daeebf2b\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"
Oct 30 22:07:07.193768 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.193747    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/957af2a6-bd01-49ed-a36a-1844daeebf2b-istio-envoy\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-xdbqw\" (UID: \"957af2a6-bd01-49ed-a36a-1844daeebf2b\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"
Oct 30 22:07:07.193823 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.193747    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a83867ab-e545-4023-8923-fc0124c9c638-istio-envoy\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-wcxns\" (UID: \"a83867ab-e545-4023-8923-fc0124c9c638\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"
Oct 30 22:07:07.193952 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.193936    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a83867ab-e545-4023-8923-fc0124c9c638-istio-podinfo\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-wcxns\" (UID: \"a83867ab-e545-4023-8923-fc0124c9c638\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"
Oct 30 22:07:07.249557 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.249455    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-crfbm\" (UniqueName: \"kubernetes.io/projected/a83867ab-e545-4023-8923-fc0124c9c638-kube-api-access-crfbm\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-wcxns\" (UID: \"a83867ab-e545-4023-8923-fc0124c9c638\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"
Oct 30 22:07:07.249928 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.249901    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a83867ab-e545-4023-8923-fc0124c9c638-istio-token\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-wcxns\" (UID: \"a83867ab-e545-4023-8923-fc0124c9c638\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"
Oct 30 22:07:07.250028 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.249946    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbc99\" (UniqueName: \"kubernetes.io/projected/957af2a6-bd01-49ed-a36a-1844daeebf2b-kube-api-access-sbc99\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-xdbqw\" (UID: \"957af2a6-bd01-49ed-a36a-1844daeebf2b\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"
Oct 30 22:07:07.250524 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.250506    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/957af2a6-bd01-49ed-a36a-1844daeebf2b-istio-token\") pod \"e2e-mt-4c260505-openshift-default-6f77466685-xdbqw\" (UID: \"957af2a6-bd01-49ed-a36a-1844daeebf2b\") " pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"
Oct 30 22:07:07.343753 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.343722    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"
Oct 30 22:07:07.361982 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.361955    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"
Oct 30 22:07:07.832534 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.832395    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"]
Oct 30 22:07:07.832876 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:07:07.832850    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod957af2a6_bd01_49ed_a36a_1844daeebf2b.slice/crio-bb4005ad98edaffbcf60552898a918d18c4dc8db58f04af501f9ad0cbd978c97 WatchSource:0}: Error finding container bb4005ad98edaffbcf60552898a918d18c4dc8db58f04af501f9ad0cbd978c97: Status 404 returned error can't find the container with id bb4005ad98edaffbcf60552898a918d18c4dc8db58f04af501f9ad0cbd978c97
Oct 30 22:07:07.837230 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.836630    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"]
Oct 30 22:07:07.839612 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.839555    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:07:07.839770 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.839748    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:07:07.848572 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.848512    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:07:07.851597 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.851562    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:07:07.851703 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.851639    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:07:07.851703 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:07.851678    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:07:08.676599 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:08.676556    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns" event={"ID":"a83867ab-e545-4023-8923-fc0124c9c638","Type":"ContainerStarted","Data":"749994a02830ffd008d548133f3d76b984fdfccfa1e442ba1cbb8c66a6d8a507"}
Oct 30 22:07:08.677166 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:08.677141    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns" event={"ID":"a83867ab-e545-4023-8923-fc0124c9c638","Type":"ContainerStarted","Data":"0e7b2802045f5ff77fe9e79451a3274ac0749ad97715c446c7dc7d308ce66a5d"}
Oct 30 22:07:08.682964 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:08.682928    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw" event={"ID":"957af2a6-bd01-49ed-a36a-1844daeebf2b","Type":"ContainerStarted","Data":"a2d4ab21d2a6b1d8cee1ea833a60a5d79c7adfc4cce028557f78d079de1228a1"}
Oct 30 22:07:08.683526 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:08.683498    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw" event={"ID":"957af2a6-bd01-49ed-a36a-1844daeebf2b","Type":"ContainerStarted","Data":"bb4005ad98edaffbcf60552898a918d18c4dc8db58f04af501f9ad0cbd978c97"}
Oct 30 22:07:08.957320 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:08.951146    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns" podStartSLOduration=2.951124188 podStartE2EDuration="2.951124188s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:07:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:08.854789499 +0000 UTC m=+1771.032278117" watchObservedRunningTime="2026-10-30 22:07:08.951124188 +0000 UTC m=+1771.128612806"
Oct 30 22:07:08.981661 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:08.981588    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw" podStartSLOduration=2.981571686 podStartE2EDuration="2.981571686s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:07:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:08.979904543 +0000 UTC m=+1771.157393162" watchObservedRunningTime="2026-10-30 22:07:08.981571686 +0000 UTC m=+1771.159060304"
Oct 30 22:07:09.362320 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:09.354540    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"
Oct 30 22:07:09.362320 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:09.354676    4871 patch_prober.go:29] interesting pod/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.128:15021/healthz/ready\": dial tcp 10.133.0.128:15021: connect: connection refused" start-of-body=
Oct 30 22:07:09.362320 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:09.354722    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw" podUID="957af2a6-bd01-49ed-a36a-1844daeebf2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.128:15021/healthz/ready\": dial tcp 10.133.0.128:15021: connect: connection refused"
Oct 30 22:07:09.376261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:09.373946    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"
Oct 30 22:07:09.376261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:09.374079    4871 patch_prober.go:29] interesting pod/e2e-mt-4c260505-openshift-default-6f77466685-wcxns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused" start-of-body=
Oct 30 22:07:09.376261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:09.374121    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns" podUID="a83867ab-e545-4023-8923-fc0124c9c638" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused"
Oct 30 22:07:09.593356 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:09.592753    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t"
Oct 30 22:07:10.346530 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:10.346495    4871 patch_prober.go:29] interesting pod/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.128:15021/healthz/ready\": dial tcp 10.133.0.128:15021: connect: connection refused" start-of-body=
Oct 30 22:07:10.347118 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:10.347085    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw" podUID="957af2a6-bd01-49ed-a36a-1844daeebf2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.128:15021/healthz/ready\": dial tcp 10.133.0.128:15021: connect: connection refused"
Oct 30 22:07:10.365828 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:10.365795    4871 patch_prober.go:29] interesting pod/e2e-mt-4c260505-openshift-default-6f77466685-wcxns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused" start-of-body=
Oct 30 22:07:10.366087 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:10.366062    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns" podUID="a83867ab-e545-4023-8923-fc0124c9c638" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused"
Oct 30 22:07:11.346260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:11.345804    4871 patch_prober.go:29] interesting pod/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.128:15021/healthz/ready\": dial tcp 10.133.0.128:15021: connect: connection refused" start-of-body=
Oct 30 22:07:11.346260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:11.345902    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw" podUID="957af2a6-bd01-49ed-a36a-1844daeebf2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.128:15021/healthz/ready\": dial tcp 10.133.0.128:15021: connect: connection refused"
Oct 30 22:07:11.362756 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:11.362716    4871 patch_prober.go:29] interesting pod/e2e-mt-4c260505-openshift-default-6f77466685-wcxns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused" start-of-body=
Oct 30 22:07:11.363133 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:11.362766    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns" podUID="a83867ab-e545-4023-8923-fc0124c9c638" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused"
Oct 30 22:07:11.457456 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:11.457399    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:12.356747 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:12.356355    4871 patch_prober.go:29] interesting pod/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.128:15021/healthz/ready\": dial tcp 10.133.0.128:15021: connect: connection refused" start-of-body=
Oct 30 22:07:12.356747 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:12.356411    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw" podUID="957af2a6-bd01-49ed-a36a-1844daeebf2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.128:15021/healthz/ready\": dial tcp 10.133.0.128:15021: connect: connection refused"
Oct 30 22:07:12.362505 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:12.362417    4871 patch_prober.go:29] interesting pod/e2e-mt-4c260505-openshift-default-6f77466685-wcxns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused" start-of-body=
Oct 30 22:07:12.362505 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:12.362467    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns" podUID="a83867ab-e545-4023-8923-fc0124c9c638" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused"
Oct 30 22:07:13.358263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:13.350329    4871 patch_prober.go:29] interesting pod/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.128:15021/healthz/ready\": dial tcp 10.133.0.128:15021: connect: connection refused" start-of-body=
Oct 30 22:07:13.358263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:13.350392    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw" podUID="957af2a6-bd01-49ed-a36a-1844daeebf2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.128:15021/healthz/ready\": dial tcp 10.133.0.128:15021: connect: connection refused"
Oct 30 22:07:13.362748 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:13.362701    4871 patch_prober.go:29] interesting pod/e2e-mt-4c260505-openshift-default-6f77466685-wcxns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused" start-of-body=
Oct 30 22:07:13.362936 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:13.362768    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns" podUID="a83867ab-e545-4023-8923-fc0124c9c638" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused"
Oct 30 22:07:14.345060 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:14.345018    4871 patch_prober.go:29] interesting pod/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.128:15021/healthz/ready\": dial tcp 10.133.0.128:15021: connect: connection refused" start-of-body=
Oct 30 22:07:14.345308 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:14.345088    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw" podUID="957af2a6-bd01-49ed-a36a-1844daeebf2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.128:15021/healthz/ready\": dial tcp 10.133.0.128:15021: connect: connection refused"
Oct 30 22:07:14.362708 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:14.362670    4871 patch_prober.go:29] interesting pod/e2e-mt-4c260505-openshift-default-6f77466685-wcxns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused" start-of-body=
Oct 30 22:07:14.363101 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:14.362737    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns" podUID="a83867ab-e545-4023-8923-fc0124c9c638" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused"
Oct 30 22:07:14.628461 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:14.628412    4871 patch_prober.go:29] interesting pod/e2e-api-b-97f097-openshift-default-589b54574c-469zm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.122:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:07:14.628678 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:14.628491    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm" podUID="d1743672-f86e-4bb4-a827-99344823cc30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.122:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:07:15.350162 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:15.350125    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"
Oct 30 22:07:15.350941 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:15.350900    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"
Oct 30 22:07:15.351947 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:15.351922    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"
Oct 30 22:07:15.369399 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:15.369368    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"
Oct 30 22:07:15.369784 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:15.369636    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"
Oct 30 22:07:15.370367 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:15.370349    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"
Oct 30 22:07:16.214117 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:16.214027    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:16.823060 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:16.822939    4871 generic.go:356] "Generic (PLEG): container finished" podID="d1743672-f86e-4bb4-a827-99344823cc30" containerID="4363fcea3e6f82ba8fc2ddac054384b0cc3df2821c66111ae176c14635aaea1c" exitCode=0
Oct 30 22:07:16.823060 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:16.823033    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm" event={"ID":"d1743672-f86e-4bb4-a827-99344823cc30","Type":"ContainerDied","Data":"4363fcea3e6f82ba8fc2ddac054384b0cc3df2821c66111ae176c14635aaea1c"}
Oct 30 22:07:16.823695 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:16.823663    4871 scope.go:122] "RemoveContainer" containerID="4363fcea3e6f82ba8fc2ddac054384b0cc3df2821c66111ae176c14635aaea1c"
Oct 30 22:07:16.825391 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:16.825365    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:07:16.825501 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:16.825433    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:07:16.825501 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:16.825466    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:07:17.848332 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:17.848287    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm" event={"ID":"d1743672-f86e-4bb4-a827-99344823cc30","Type":"ContainerStarted","Data":"542057cf992ca8ac9b971afd54855ba1ca9dabef129ae6484adc5a9d46e95c78"}
Oct 30 22:07:18.849768 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:18.849722    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"
Oct 30 22:07:18.850305 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:18.849896    4871 patch_prober.go:29] interesting pod/e2e-api-b-97f097-openshift-default-589b54574c-469zm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused" start-of-body=
Oct 30 22:07:18.850305 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:18.849959    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm" podUID="d1743672-f86e-4bb4-a827-99344823cc30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused"
Oct 30 22:07:19.847604 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:19.847567    4871 patch_prober.go:29] interesting pod/e2e-api-b-97f097-openshift-default-589b54574c-469zm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused" start-of-body=
Oct 30 22:07:19.847812 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:19.847636    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm" podUID="d1743672-f86e-4bb4-a827-99344823cc30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused"
Oct 30 22:07:20.847821 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:20.847778    4871 patch_prober.go:29] interesting pod/e2e-api-b-97f097-openshift-default-589b54574c-469zm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused" start-of-body=
Oct 30 22:07:20.848487 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:20.848435    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm" podUID="d1743672-f86e-4bb4-a827-99344823cc30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused"
Oct 30 22:07:21.152891 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:21.152859    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:21.847021 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:21.846974    4871 patch_prober.go:29] interesting pod/e2e-api-b-97f097-openshift-default-589b54574c-469zm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused" start-of-body=
Oct 30 22:07:21.847213 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:21.847063    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm" podUID="d1743672-f86e-4bb4-a827-99344823cc30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused"
Oct 30 22:07:22.847437 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:22.847395    4871 patch_prober.go:29] interesting pod/e2e-api-b-97f097-openshift-default-589b54574c-469zm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused" start-of-body=
Oct 30 22:07:22.847837 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:22.847474    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm" podUID="d1743672-f86e-4bb4-a827-99344823cc30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused"
Oct 30 22:07:23.852085 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:23.852051    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"
Oct 30 22:07:23.852593 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:23.852574    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"
Oct 30 22:07:23.853454 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:23.853433    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"
Oct 30 22:07:25.477289 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:25.477253    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-695f5c-744b8495dc-smjpj"]
Oct 30 22:07:25.478290 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:25.478186    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-695f5c-744b8495dc-smjpj" podUID="9ea1b9a4-cdb1-475d-82f4-7b286e8410dc" containerName="maas-api" containerID="cri-o://d12ee8e00618ec10b114ead08dc778eaf4d3ae2a17fb892d08e60a098853232f" gracePeriod=30
Oct 30 22:07:25.716295 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:25.698919    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:26.269302 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:26.269214    4871 generic.go:356] "Generic (PLEG): container finished" podID="9ea1b9a4-cdb1-475d-82f4-7b286e8410dc" containerID="d12ee8e00618ec10b114ead08dc778eaf4d3ae2a17fb892d08e60a098853232f" exitCode=0
Oct 30 22:07:26.269723 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:26.269691    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-695f5c-744b8495dc-smjpj" event={"ID":"9ea1b9a4-cdb1-475d-82f4-7b286e8410dc","Type":"ContainerDied","Data":"d12ee8e00618ec10b114ead08dc778eaf4d3ae2a17fb892d08e60a098853232f"}
Oct 30 22:07:26.283265 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:26.280860    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-695f5c-744b8495dc-smjpj" podUID="9ea1b9a4-cdb1-475d-82f4-7b286e8410dc" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.114:8443/health\": dial tcp 10.133.0.114:8443: connect: connection refused"
Oct 30 22:07:26.524875 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:26.524837    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-695f5c-744b8495dc-smjpj"
Oct 30 22:07:26.728554 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:26.727157    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9ea1b9a4-cdb1-475d-82f4-7b286e8410dc-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ea1b9a4-cdb1-475d-82f4-7b286e8410dc-maas-api-tls\") pod \"9ea1b9a4-cdb1-475d-82f4-7b286e8410dc\" (UID: \"9ea1b9a4-cdb1-475d-82f4-7b286e8410dc\") "
Oct 30 22:07:26.728554 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:26.727317    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9ea1b9a4-cdb1-475d-82f4-7b286e8410dc-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ea1b9a4-cdb1-475d-82f4-7b286e8410dc-openshift-service-ca\") pod \"9ea1b9a4-cdb1-475d-82f4-7b286e8410dc\" (UID: \"9ea1b9a4-cdb1-475d-82f4-7b286e8410dc\") "
Oct 30 22:07:26.728554 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:26.727461    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9ea1b9a4-cdb1-475d-82f4-7b286e8410dc-metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ea1b9a4-cdb1-475d-82f4-7b286e8410dc-metrics-certs\") pod \"9ea1b9a4-cdb1-475d-82f4-7b286e8410dc\" (UID: \"9ea1b9a4-cdb1-475d-82f4-7b286e8410dc\") "
Oct 30 22:07:26.728554 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:26.727513    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9ea1b9a4-cdb1-475d-82f4-7b286e8410dc-kube-api-access-42z79\" (UniqueName: \"kubernetes.io/projected/9ea1b9a4-cdb1-475d-82f4-7b286e8410dc-kube-api-access-42z79\") pod \"9ea1b9a4-cdb1-475d-82f4-7b286e8410dc\" (UID: \"9ea1b9a4-cdb1-475d-82f4-7b286e8410dc\") "
Oct 30 22:07:26.750203 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:26.749466    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ea1b9a4-cdb1-475d-82f4-7b286e8410dc-openshift-service-ca" pod "9ea1b9a4-cdb1-475d-82f4-7b286e8410dc" (UID: "9ea1b9a4-cdb1-475d-82f4-7b286e8410dc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:07:26.776310 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:26.754856    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ea1b9a4-cdb1-475d-82f4-7b286e8410dc-kube-api-access-42z79" pod "9ea1b9a4-cdb1-475d-82f4-7b286e8410dc" (UID: "9ea1b9a4-cdb1-475d-82f4-7b286e8410dc"). InnerVolumeSpecName "kube-api-access-42z79". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:07:26.818823 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:26.815935    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ea1b9a4-cdb1-475d-82f4-7b286e8410dc-maas-api-tls" pod "9ea1b9a4-cdb1-475d-82f4-7b286e8410dc" (UID: "9ea1b9a4-cdb1-475d-82f4-7b286e8410dc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:07:26.869559 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:26.864834    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42z79\" (UniqueName: \"kubernetes.io/projected/9ea1b9a4-cdb1-475d-82f4-7b286e8410dc-kube-api-access-42z79\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:26.869559 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:26.864877    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ea1b9a4-cdb1-475d-82f4-7b286e8410dc-maas-api-tls\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:26.869559 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:26.864892    4871 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ea1b9a4-cdb1-475d-82f4-7b286e8410dc-openshift-service-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:26.874146 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:26.874105    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ea1b9a4-cdb1-475d-82f4-7b286e8410dc-metrics-certs" pod "9ea1b9a4-cdb1-475d-82f4-7b286e8410dc" (UID: "9ea1b9a4-cdb1-475d-82f4-7b286e8410dc"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:07:26.966927 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:26.966888    4871 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ea1b9a4-cdb1-475d-82f4-7b286e8410dc-metrics-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:27.279121 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:27.279088    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-695f5c-744b8495dc-smjpj"
Oct 30 22:07:27.279121 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:27.279099    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-695f5c-744b8495dc-smjpj" event={"ID":"9ea1b9a4-cdb1-475d-82f4-7b286e8410dc","Type":"ContainerDied","Data":"504e70587186f0ef498cb24181d111d1883592e7d5037ba4562906f6a989f9ed"}
Oct 30 22:07:27.279417 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:27.279204    4871 scope.go:122] "RemoveContainer" containerID="d12ee8e00618ec10b114ead08dc778eaf4d3ae2a17fb892d08e60a098853232f"
Oct 30 22:07:27.430829 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:27.430791    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:27.492671 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:27.492635    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-695f5c-744b8495dc-smjpj"]
Oct 30 22:07:27.544509 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:27.544411    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-695f5c-744b8495dc-smjpj"]
Oct 30 22:07:27.739846 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:27.739811    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:27.926840 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:27.926809    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:27.962051 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:27.962017    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9ea1b9a4-cdb1-475d-82f4-7b286e8410dc" path="/var/lib/kubelet/pods/9ea1b9a4-cdb1-475d-82f4-7b286e8410dc/volumes"
Oct 30 22:07:28.377693 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:28.377660    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:28.551193 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:28.551152    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:31.278316 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:31.278276    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:32.819837 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:32.819800    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:34.663485 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:34.663448    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:34.814295 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:34.814218    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:35.017835 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:35.017726    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"]
Oct 30 22:07:35.018449 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:35.018412    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272" podUID="48a6bcac-9719-4358-9019-71fd0a388ba1" containerName="istio-proxy" containerID="cri-o://9809017f93fd0d95d4b865dafea8cc1b1073da5111a4332a8c88bba6a6aac440" gracePeriod=30
Oct 30 22:07:35.040722 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:35.040688    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:35.577518 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:07:35.577463    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:07:37.785313 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:37.785230    4871 patch_prober.go:29] interesting pod/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": read tcp 10.133.0.2:44104->10.133.0.124:15021: read: connection reset by peer" start-of-body=
Oct 30 22:07:37.785848 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:37.785336    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh" podUID="42c86b8e-04b6-49fa-af33-9bc73297d162" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": read tcp 10.133.0.2:44104->10.133.0.124:15021: read: connection reset by peer"
Oct 30 22:07:37.846306 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:37.846232    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:38.270034 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:38.270007    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx_8d401654-8a4c-4a79-b2d3-612765312421/extract/0.log"
Oct 30 22:07:38.278597 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:38.278572    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx_8d401654-8a4c-4a79-b2d3-612765312421/extract/0.log"
Oct 30 22:07:38.282927 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:38.282905    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/2.log"
Oct 30 22:07:38.291364 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:38.291337    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzntd_2360fddc-52a5-42a5-82ca-a05aff84d217/ovn-acl-logging/0.log"
Oct 30 22:07:38.292324 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:38.292305    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/2.log"
Oct 30 22:07:38.300756 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:38.300732    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzntd_2360fddc-52a5-42a5-82ca-a05aff84d217/ovn-acl-logging/0.log"
Oct 30 22:07:38.491638 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:38.491604    4871 generic.go:356] "Generic (PLEG): container finished" podID="42c86b8e-04b6-49fa-af33-9bc73297d162" containerID="0f027d9405f9db17345e77c83cb859288cbc11f8f45e5c9c2349f0487b3bcb21" exitCode=0
Oct 30 22:07:38.491809 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:38.491671    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh" event={"ID":"42c86b8e-04b6-49fa-af33-9bc73297d162","Type":"ContainerDied","Data":"0f027d9405f9db17345e77c83cb859288cbc11f8f45e5c9c2349f0487b3bcb21"}
Oct 30 22:07:38.492128 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:38.492114    4871 scope.go:122] "RemoveContainer" containerID="0f027d9405f9db17345e77c83cb859288cbc11f8f45e5c9c2349f0487b3bcb21"
Oct 30 22:07:38.493821 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:38.493794    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:07:38.493882 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:38.493871    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:07:38.493920 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:38.493904    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:07:38.614091 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:38.614056    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:39.502052 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:39.501967    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh" event={"ID":"42c86b8e-04b6-49fa-af33-9bc73297d162","Type":"ContainerStarted","Data":"0f3e66340d43693a333b872b1a92745111c02cd543488d0ed788eb3e199960a0"}
Oct 30 22:07:39.551144 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:39.551109    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"
Oct 30 22:07:39.551403 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:39.551336    4871 patch_prober.go:29] interesting pod/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused" start-of-body=
Oct 30 22:07:39.551498 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:39.551408    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh" podUID="42c86b8e-04b6-49fa-af33-9bc73297d162" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused"
Oct 30 22:07:39.647656 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:39.647622    4871 patch_prober.go:29] interesting pod/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:07:39.647823 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:39.647676    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272" podUID="48a6bcac-9719-4358-9019-71fd0a388ba1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:07:40.355304 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.355267    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"
Oct 30 22:07:40.374899 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.374863    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48a6bcac-9719-4358-9019-71fd0a388ba1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48a6bcac-9719-4358-9019-71fd0a388ba1-istio-data\") pod \"48a6bcac-9719-4358-9019-71fd0a388ba1\" (UID: \"48a6bcac-9719-4358-9019-71fd0a388ba1\") "
Oct 30 22:07:40.374899 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.374904    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/48a6bcac-9719-4358-9019-71fd0a388ba1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48a6bcac-9719-4358-9019-71fd0a388ba1-istiod-ca-cert\") pod \"48a6bcac-9719-4358-9019-71fd0a388ba1\" (UID: \"48a6bcac-9719-4358-9019-71fd0a388ba1\") "
Oct 30 22:07:40.375162 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.374924    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48a6bcac-9719-4358-9019-71fd0a388ba1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48a6bcac-9719-4358-9019-71fd0a388ba1-credential-socket\") pod \"48a6bcac-9719-4358-9019-71fd0a388ba1\" (UID: \"48a6bcac-9719-4358-9019-71fd0a388ba1\") "
Oct 30 22:07:40.375162 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.375093    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48a6bcac-9719-4358-9019-71fd0a388ba1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48a6bcac-9719-4358-9019-71fd0a388ba1-workload-socket\") pod \"48a6bcac-9719-4358-9019-71fd0a388ba1\" (UID: \"48a6bcac-9719-4358-9019-71fd0a388ba1\") "
Oct 30 22:07:40.375162 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.375120    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/48a6bcac-9719-4358-9019-71fd0a388ba1-kube-api-access-5xk8p\" (UniqueName: \"kubernetes.io/projected/48a6bcac-9719-4358-9019-71fd0a388ba1-kube-api-access-5xk8p\") pod \"48a6bcac-9719-4358-9019-71fd0a388ba1\" (UID: \"48a6bcac-9719-4358-9019-71fd0a388ba1\") "
Oct 30 22:07:40.375162 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.375148    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48a6bcac-9719-4358-9019-71fd0a388ba1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48a6bcac-9719-4358-9019-71fd0a388ba1-istio-envoy\") pod \"48a6bcac-9719-4358-9019-71fd0a388ba1\" (UID: \"48a6bcac-9719-4358-9019-71fd0a388ba1\") "
Oct 30 22:07:40.375401 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.375211    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/48a6bcac-9719-4358-9019-71fd0a388ba1-istio-token\" (UniqueName: \"kubernetes.io/projected/48a6bcac-9719-4358-9019-71fd0a388ba1-istio-token\") pod \"48a6bcac-9719-4358-9019-71fd0a388ba1\" (UID: \"48a6bcac-9719-4358-9019-71fd0a388ba1\") "
Oct 30 22:07:40.375401 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.375259    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48a6bcac-9719-4358-9019-71fd0a388ba1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48a6bcac-9719-4358-9019-71fd0a388ba1-workload-certs\") pod \"48a6bcac-9719-4358-9019-71fd0a388ba1\" (UID: \"48a6bcac-9719-4358-9019-71fd0a388ba1\") "
Oct 30 22:07:40.375401 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.375300    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/48a6bcac-9719-4358-9019-71fd0a388ba1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48a6bcac-9719-4358-9019-71fd0a388ba1-istio-podinfo\") pod \"48a6bcac-9719-4358-9019-71fd0a388ba1\" (UID: \"48a6bcac-9719-4358-9019-71fd0a388ba1\") "
Oct 30 22:07:40.375401 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.375364    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48a6bcac-9719-4358-9019-71fd0a388ba1-credential-socket" pod "48a6bcac-9719-4358-9019-71fd0a388ba1" (UID: "48a6bcac-9719-4358-9019-71fd0a388ba1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:07:40.375601 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.375393    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48a6bcac-9719-4358-9019-71fd0a388ba1-istiod-ca-cert" pod "48a6bcac-9719-4358-9019-71fd0a388ba1" (UID: "48a6bcac-9719-4358-9019-71fd0a388ba1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:07:40.375831 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.375798    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48a6bcac-9719-4358-9019-71fd0a388ba1-workload-socket" pod "48a6bcac-9719-4358-9019-71fd0a388ba1" (UID: "48a6bcac-9719-4358-9019-71fd0a388ba1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:07:40.375908 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.375863    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48a6bcac-9719-4358-9019-71fd0a388ba1-istio-data" pod "48a6bcac-9719-4358-9019-71fd0a388ba1" (UID: "48a6bcac-9719-4358-9019-71fd0a388ba1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:07:40.375967 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.375907    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48a6bcac-9719-4358-9019-71fd0a388ba1-workload-certs" pod "48a6bcac-9719-4358-9019-71fd0a388ba1" (UID: "48a6bcac-9719-4358-9019-71fd0a388ba1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:07:40.376543 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.376516    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48a6bcac-9719-4358-9019-71fd0a388ba1-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:40.376633 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.376548    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48a6bcac-9719-4358-9019-71fd0a388ba1-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:40.376633 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.376564    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48a6bcac-9719-4358-9019-71fd0a388ba1-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:40.376633 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.376580    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48a6bcac-9719-4358-9019-71fd0a388ba1-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:40.376633 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.376595    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48a6bcac-9719-4358-9019-71fd0a388ba1-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:40.377917 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.377889    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48a6bcac-9719-4358-9019-71fd0a388ba1-kube-api-access-5xk8p" pod "48a6bcac-9719-4358-9019-71fd0a388ba1" (UID: "48a6bcac-9719-4358-9019-71fd0a388ba1"). InnerVolumeSpecName "kube-api-access-5xk8p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:07:40.378024 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.378004    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48a6bcac-9719-4358-9019-71fd0a388ba1-istio-envoy" pod "48a6bcac-9719-4358-9019-71fd0a388ba1" (UID: "48a6bcac-9719-4358-9019-71fd0a388ba1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:07:40.378094 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.378072    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/48a6bcac-9719-4358-9019-71fd0a388ba1-istio-podinfo" pod "48a6bcac-9719-4358-9019-71fd0a388ba1" (UID: "48a6bcac-9719-4358-9019-71fd0a388ba1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:07:40.378129 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.378098    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48a6bcac-9719-4358-9019-71fd0a388ba1-istio-token" pod "48a6bcac-9719-4358-9019-71fd0a388ba1" (UID: "48a6bcac-9719-4358-9019-71fd0a388ba1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:07:40.478491 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.478453    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5xk8p\" (UniqueName: \"kubernetes.io/projected/48a6bcac-9719-4358-9019-71fd0a388ba1-kube-api-access-5xk8p\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:40.478491 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.478493    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48a6bcac-9719-4358-9019-71fd0a388ba1-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:40.478690 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.478508    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48a6bcac-9719-4358-9019-71fd0a388ba1-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:40.478690 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.478526    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48a6bcac-9719-4358-9019-71fd0a388ba1-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:40.543801 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.543762    4871 generic.go:356] "Generic (PLEG): container finished" podID="48a6bcac-9719-4358-9019-71fd0a388ba1" containerID="9809017f93fd0d95d4b865dafea8cc1b1073da5111a4332a8c88bba6a6aac440" exitCode=0
Oct 30 22:07:40.544328 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.543832    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"
Oct 30 22:07:40.544328 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.543997    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272" event={"ID":"48a6bcac-9719-4358-9019-71fd0a388ba1","Type":"ContainerDied","Data":"9809017f93fd0d95d4b865dafea8cc1b1073da5111a4332a8c88bba6a6aac440"}
Oct 30 22:07:40.544328 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.544051    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272" event={"ID":"48a6bcac-9719-4358-9019-71fd0a388ba1","Type":"ContainerDied","Data":"a658cca2940a68835afc76ca8042275fb0f5d155bb294a25a103f112b1ad18bd"}
Oct 30 22:07:40.544328 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.544082    4871 scope.go:122] "RemoveContainer" containerID="9809017f93fd0d95d4b865dafea8cc1b1073da5111a4332a8c88bba6a6aac440"
Oct 30 22:07:40.551630 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.551568    4871 patch_prober.go:29] interesting pod/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused" start-of-body=
Oct 30 22:07:40.551630 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.551617    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh" podUID="42c86b8e-04b6-49fa-af33-9bc73297d162" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused"
Oct 30 22:07:40.561592 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.561570    4871 scope.go:122] "RemoveContainer" containerID="fa7716b450593c2f234b240a01087cdb064d22e8b477c568e49d862e0549c63f"
Oct 30 22:07:40.573558 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.573535    4871 scope.go:122] "RemoveContainer" containerID="9809017f93fd0d95d4b865dafea8cc1b1073da5111a4332a8c88bba6a6aac440"
Oct 30 22:07:40.573950 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:07:40.573928    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9809017f93fd0d95d4b865dafea8cc1b1073da5111a4332a8c88bba6a6aac440\": container with ID starting with 9809017f93fd0d95d4b865dafea8cc1b1073da5111a4332a8c88bba6a6aac440 not found: ID does not exist" containerID="9809017f93fd0d95d4b865dafea8cc1b1073da5111a4332a8c88bba6a6aac440"
Oct 30 22:07:40.574017 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.573962    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9809017f93fd0d95d4b865dafea8cc1b1073da5111a4332a8c88bba6a6aac440"} err="failed to get container status \"9809017f93fd0d95d4b865dafea8cc1b1073da5111a4332a8c88bba6a6aac440\": rpc error: code = NotFound desc = could not find container \"9809017f93fd0d95d4b865dafea8cc1b1073da5111a4332a8c88bba6a6aac440\": container with ID starting with 9809017f93fd0d95d4b865dafea8cc1b1073da5111a4332a8c88bba6a6aac440 not found: ID does not exist"
Oct 30 22:07:40.574017 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.573987    4871 scope.go:122] "RemoveContainer" containerID="fa7716b450593c2f234b240a01087cdb064d22e8b477c568e49d862e0549c63f"
Oct 30 22:07:40.574336 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:07:40.574309    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa7716b450593c2f234b240a01087cdb064d22e8b477c568e49d862e0549c63f\": container with ID starting with fa7716b450593c2f234b240a01087cdb064d22e8b477c568e49d862e0549c63f not found: ID does not exist" containerID="fa7716b450593c2f234b240a01087cdb064d22e8b477c568e49d862e0549c63f"
Oct 30 22:07:40.574403 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.574342    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa7716b450593c2f234b240a01087cdb064d22e8b477c568e49d862e0549c63f"} err="failed to get container status \"fa7716b450593c2f234b240a01087cdb064d22e8b477c568e49d862e0549c63f\": rpc error: code = NotFound desc = could not find container \"fa7716b450593c2f234b240a01087cdb064d22e8b477c568e49d862e0549c63f\": container with ID starting with fa7716b450593c2f234b240a01087cdb064d22e8b477c568e49d862e0549c63f not found: ID does not exist"
Oct 30 22:07:40.733678 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.733640    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"]
Oct 30 22:07:40.739759 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:40.739723    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-695f5c-openshift-default-94db46fbf-9c272"]
Oct 30 22:07:41.553026 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:41.552989    4871 patch_prober.go:29] interesting pod/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused" start-of-body=
Oct 30 22:07:41.553541 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:41.553061    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh" podUID="42c86b8e-04b6-49fa-af33-9bc73297d162" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused"
Oct 30 22:07:41.961962 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:41.961925    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="48a6bcac-9719-4358-9019-71fd0a388ba1" path="/var/lib/kubelet/pods/48a6bcac-9719-4358-9019-71fd0a388ba1/volumes"
Oct 30 22:07:42.551128 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:42.551093    4871 patch_prober.go:29] interesting pod/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused" start-of-body=
Oct 30 22:07:42.551360 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:42.551154    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh" podUID="42c86b8e-04b6-49fa-af33-9bc73297d162" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused"
Oct 30 22:07:42.781286 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:42.781222    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-xtg4m"]
Oct 30 22:07:42.853299 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:42.853213    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:43.164035 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:43.163989    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:43.350270 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:43.331800    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:43.551825 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:43.551736    4871 patch_prober.go:29] interesting pod/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused" start-of-body=
Oct 30 22:07:43.551977 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:43.551812    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh" podUID="42c86b8e-04b6-49fa-af33-9bc73297d162" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused"
Oct 30 22:07:44.561835 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:44.554570    4871 patch_prober.go:29] interesting pod/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused" start-of-body=
Oct 30 22:07:44.561835 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:44.554643    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh" podUID="42c86b8e-04b6-49fa-af33-9bc73297d162" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused"
Oct 30 22:07:45.558538 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:45.558492    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"
Oct 30 22:07:45.559710 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:45.559678    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"
Oct 30 22:07:45.565724 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:45.565694    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"
Oct 30 22:07:47.253562 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:47.253528    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk"]
Oct 30 22:07:47.253996 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:47.253729    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk" podUID="f61805de-c5e9-42ef-9382-1e1c5c7d188e" containerName="maas-api" containerID="cri-o://4b5fbe8f2698d37894f279abdaab680d9b5e6225f7d1679f2d824535b05dc556" gracePeriod=30
Oct 30 22:07:47.371601 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:47.371573    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:47.600367 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:47.600333    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk"
Oct 30 22:07:47.681889 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:47.681852    4871 generic.go:356] "Generic (PLEG): container finished" podID="f61805de-c5e9-42ef-9382-1e1c5c7d188e" containerID="4b5fbe8f2698d37894f279abdaab680d9b5e6225f7d1679f2d824535b05dc556" exitCode=0
Oct 30 22:07:47.682153 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:47.682131    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk" event={"ID":"f61805de-c5e9-42ef-9382-1e1c5c7d188e","Type":"ContainerDied","Data":"4b5fbe8f2698d37894f279abdaab680d9b5e6225f7d1679f2d824535b05dc556"}
Oct 30 22:07:47.682323 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:47.682304    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk" event={"ID":"f61805de-c5e9-42ef-9382-1e1c5c7d188e","Type":"ContainerDied","Data":"94843748c19afe0e4c51c9d088fa8dca109c876c35d720a47fdd42e62dcb8574"}
Oct 30 22:07:47.682456 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:47.682443    4871 scope.go:122] "RemoveContainer" containerID="4b5fbe8f2698d37894f279abdaab680d9b5e6225f7d1679f2d824535b05dc556"
Oct 30 22:07:47.682701 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:47.682685    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk"
Oct 30 22:07:47.703777 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:47.703747    4871 scope.go:122] "RemoveContainer" containerID="4b5fbe8f2698d37894f279abdaab680d9b5e6225f7d1679f2d824535b05dc556"
Oct 30 22:07:47.704120 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:07:47.704093    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b5fbe8f2698d37894f279abdaab680d9b5e6225f7d1679f2d824535b05dc556\": container with ID starting with 4b5fbe8f2698d37894f279abdaab680d9b5e6225f7d1679f2d824535b05dc556 not found: ID does not exist" containerID="4b5fbe8f2698d37894f279abdaab680d9b5e6225f7d1679f2d824535b05dc556"
Oct 30 22:07:47.704190 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:47.704133    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b5fbe8f2698d37894f279abdaab680d9b5e6225f7d1679f2d824535b05dc556"} err="failed to get container status \"4b5fbe8f2698d37894f279abdaab680d9b5e6225f7d1679f2d824535b05dc556\": rpc error: code = NotFound desc = could not find container \"4b5fbe8f2698d37894f279abdaab680d9b5e6225f7d1679f2d824535b05dc556\": container with ID starting with 4b5fbe8f2698d37894f279abdaab680d9b5e6225f7d1679f2d824535b05dc556 not found: ID does not exist"
Oct 30 22:07:47.753102 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:47.753065    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:47.781812 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:47.781725    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f61805de-c5e9-42ef-9382-1e1c5c7d188e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f61805de-c5e9-42ef-9382-1e1c5c7d188e-metrics-certs\") pod \"f61805de-c5e9-42ef-9382-1e1c5c7d188e\" (UID: \"f61805de-c5e9-42ef-9382-1e1c5c7d188e\") "
Oct 30 22:07:47.782116 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:47.782092    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f61805de-c5e9-42ef-9382-1e1c5c7d188e-kube-api-access-5zdqk\" (UniqueName: \"kubernetes.io/projected/f61805de-c5e9-42ef-9382-1e1c5c7d188e-kube-api-access-5zdqk\") pod \"f61805de-c5e9-42ef-9382-1e1c5c7d188e\" (UID: \"f61805de-c5e9-42ef-9382-1e1c5c7d188e\") "
Oct 30 22:07:47.782375 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:47.782355    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f61805de-c5e9-42ef-9382-1e1c5c7d188e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f61805de-c5e9-42ef-9382-1e1c5c7d188e-openshift-service-ca\") pod \"f61805de-c5e9-42ef-9382-1e1c5c7d188e\" (UID: \"f61805de-c5e9-42ef-9382-1e1c5c7d188e\") "
Oct 30 22:07:47.782566 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:47.781763    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-xtg4m" podUID="bc280bff-3593-4ae0-b27d-88339a1e100b" containerName="payload-processing" containerID="cri-o://9809df9d7bc34e79c33e0ea13934e306fa27f9b65c2ad3d5b7d6d395e8b423ad" gracePeriod=25
Oct 30 22:07:47.783323 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:47.782803    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f61805de-c5e9-42ef-9382-1e1c5c7d188e-openshift-service-ca" pod "f61805de-c5e9-42ef-9382-1e1c5c7d188e" (UID: "f61805de-c5e9-42ef-9382-1e1c5c7d188e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:07:47.783760 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:47.783742    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f61805de-c5e9-42ef-9382-1e1c5c7d188e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f61805de-c5e9-42ef-9382-1e1c5c7d188e-maas-api-tls\") pod \"f61805de-c5e9-42ef-9382-1e1c5c7d188e\" (UID: \"f61805de-c5e9-42ef-9382-1e1c5c7d188e\") "
Oct 30 22:07:47.784846 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:47.784800    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f61805de-c5e9-42ef-9382-1e1c5c7d188e-metrics-certs" pod "f61805de-c5e9-42ef-9382-1e1c5c7d188e" (UID: "f61805de-c5e9-42ef-9382-1e1c5c7d188e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:07:47.785088 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:47.785060    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f61805de-c5e9-42ef-9382-1e1c5c7d188e-kube-api-access-5zdqk" pod "f61805de-c5e9-42ef-9382-1e1c5c7d188e" (UID: "f61805de-c5e9-42ef-9382-1e1c5c7d188e"). InnerVolumeSpecName "kube-api-access-5zdqk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:07:47.785213 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:47.785192    4871 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f61805de-c5e9-42ef-9382-1e1c5c7d188e-metrics-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:47.785395 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:47.785380    4871 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f61805de-c5e9-42ef-9382-1e1c5c7d188e-openshift-service-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:47.787164 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:47.787136    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f61805de-c5e9-42ef-9382-1e1c5c7d188e-maas-api-tls" pod "f61805de-c5e9-42ef-9382-1e1c5c7d188e" (UID: "f61805de-c5e9-42ef-9382-1e1c5c7d188e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:07:47.901701 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:47.888608    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5zdqk\" (UniqueName: \"kubernetes.io/projected/f61805de-c5e9-42ef-9382-1e1c5c7d188e-kube-api-access-5zdqk\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:47.901701 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:47.888663    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f61805de-c5e9-42ef-9382-1e1c5c7d188e-maas-api-tls\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:47.913258 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:47.910227    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:48.145533 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:48.145455    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk"]
Oct 30 22:07:48.150548 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:48.150517    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-2d2f63-67498dc576-5gjgk"]
Oct 30 22:07:48.182007 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:48.181974    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-xtg4m"
Oct 30 22:07:48.205381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:48.205343    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bc280bff-3593-4ae0-b27d-88339a1e100b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc280bff-3593-4ae0-b27d-88339a1e100b-plugins-config-volume\") pod \"bc280bff-3593-4ae0-b27d-88339a1e100b\" (UID: \"bc280bff-3593-4ae0-b27d-88339a1e100b\") "
Oct 30 22:07:48.205593 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:48.205517    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bc280bff-3593-4ae0-b27d-88339a1e100b-kube-api-access-zfwcn\" (UniqueName: \"kubernetes.io/projected/bc280bff-3593-4ae0-b27d-88339a1e100b-kube-api-access-zfwcn\") pod \"bc280bff-3593-4ae0-b27d-88339a1e100b\" (UID: \"bc280bff-3593-4ae0-b27d-88339a1e100b\") "
Oct 30 22:07:48.205796 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:48.205771    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc280bff-3593-4ae0-b27d-88339a1e100b-plugins-config-volume" pod "bc280bff-3593-4ae0-b27d-88339a1e100b" (UID: "bc280bff-3593-4ae0-b27d-88339a1e100b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:07:48.206014 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:48.205997    4871 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc280bff-3593-4ae0-b27d-88339a1e100b-plugins-config-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:48.207700 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:48.207676    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc280bff-3593-4ae0-b27d-88339a1e100b-kube-api-access-zfwcn" pod "bc280bff-3593-4ae0-b27d-88339a1e100b" (UID: "bc280bff-3593-4ae0-b27d-88339a1e100b"). InnerVolumeSpecName "kube-api-access-zfwcn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:07:48.329311 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:48.318229    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zfwcn\" (UniqueName: \"kubernetes.io/projected/bc280bff-3593-4ae0-b27d-88339a1e100b-kube-api-access-zfwcn\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:48.595902 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:48.595818    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"]
Oct 30 22:07:48.596089 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:48.596066    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h" podUID="b28e1c99-b48d-4a4e-a225-ebba779327e4" containerName="istio-proxy" containerID="cri-o://c4df035ff8ad9aa06e79ea2ff334fd4a2b3b149452677e5223043972868bed19" gracePeriod=30
Oct 30 22:07:48.601767 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:48.601736    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"]
Oct 30 22:07:48.602021 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:48.601990    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j" podUID="19911666-f25a-4dd7-9789-7b9374b61cfb" containerName="istio-proxy" containerID="cri-o://af426aeb828c2148a3ba12ce18f7495dc345415ee0419868e346a97bb8a09a8f" gracePeriod=30
Oct 30 22:07:48.644317 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:48.644230    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"]
Oct 30 22:07:48.644556 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:48.644503    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8" podUID="ffc32abd-61d0-447a-9050-2ad518ca585f" containerName="istio-proxy" containerID="cri-o://74b1a584476cfaea2631fc7f42dd12b0cf532676d1098a88eb9a215d8bb7905b" gracePeriod=30
Oct 30 22:07:48.693611 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:48.693573    4871 generic.go:356] "Generic (PLEG): container finished" podID="bc280bff-3593-4ae0-b27d-88339a1e100b" containerID="9809df9d7bc34e79c33e0ea13934e306fa27f9b65c2ad3d5b7d6d395e8b423ad" exitCode=0
Oct 30 22:07:48.693820 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:48.693622    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-xtg4m"
Oct 30 22:07:48.693820 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:48.693661    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-xtg4m" event={"ID":"bc280bff-3593-4ae0-b27d-88339a1e100b","Type":"ContainerDied","Data":"9809df9d7bc34e79c33e0ea13934e306fa27f9b65c2ad3d5b7d6d395e8b423ad"}
Oct 30 22:07:48.694076 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:48.694046    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-xtg4m" event={"ID":"bc280bff-3593-4ae0-b27d-88339a1e100b","Type":"ContainerDied","Data":"f2e95bd8729302fdf1ebdced36ae7140d125ee229421abf5261a158c3f3d3234"}
Oct 30 22:07:48.694220 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:48.694095    4871 scope.go:122] "RemoveContainer" containerID="9809df9d7bc34e79c33e0ea13934e306fa27f9b65c2ad3d5b7d6d395e8b423ad"
Oct 30 22:07:48.707481 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:48.707463    4871 scope.go:122] "RemoveContainer" containerID="9809df9d7bc34e79c33e0ea13934e306fa27f9b65c2ad3d5b7d6d395e8b423ad"
Oct 30 22:07:48.707785 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:07:48.707766    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9809df9d7bc34e79c33e0ea13934e306fa27f9b65c2ad3d5b7d6d395e8b423ad\": container with ID starting with 9809df9d7bc34e79c33e0ea13934e306fa27f9b65c2ad3d5b7d6d395e8b423ad not found: ID does not exist" containerID="9809df9d7bc34e79c33e0ea13934e306fa27f9b65c2ad3d5b7d6d395e8b423ad"
Oct 30 22:07:48.707837 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:48.707792    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9809df9d7bc34e79c33e0ea13934e306fa27f9b65c2ad3d5b7d6d395e8b423ad"} err="failed to get container status \"9809df9d7bc34e79c33e0ea13934e306fa27f9b65c2ad3d5b7d6d395e8b423ad\": rpc error: code = NotFound desc = could not find container \"9809df9d7bc34e79c33e0ea13934e306fa27f9b65c2ad3d5b7d6d395e8b423ad\": container with ID starting with 9809df9d7bc34e79c33e0ea13934e306fa27f9b65c2ad3d5b7d6d395e8b423ad not found: ID does not exist"
Oct 30 22:07:49.191261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:49.181298    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-xtg4m"]
Oct 30 22:07:49.260263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:49.255864    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4c260505-68cf845c4f-xtg4m"]
Oct 30 22:07:49.441317 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:49.440190    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:49.754464 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:49.754383    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:49.912990 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:49.912958    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:49.963476 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:49.963445    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bc280bff-3593-4ae0-b27d-88339a1e100b" path="/var/lib/kubelet/pods/bc280bff-3593-4ae0-b27d-88339a1e100b/volumes"
Oct 30 22:07:49.963990 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:49.963975    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f61805de-c5e9-42ef-9382-1e1c5c7d188e" path="/var/lib/kubelet/pods/f61805de-c5e9-42ef-9382-1e1c5c7d188e/volumes"
Oct 30 22:07:50.411933 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:50.411893    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:52.832292 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:52.832233    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:52.999535 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:52.998442    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:53.741957 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:53.741918    4871 generic.go:356] "Generic (PLEG): container finished" podID="19911666-f25a-4dd7-9789-7b9374b61cfb" containerID="af426aeb828c2148a3ba12ce18f7495dc345415ee0419868e346a97bb8a09a8f" exitCode=0
Oct 30 22:07:53.742149 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:53.741956    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j" event={"ID":"19911666-f25a-4dd7-9789-7b9374b61cfb","Type":"ContainerDied","Data":"af426aeb828c2148a3ba12ce18f7495dc345415ee0419868e346a97bb8a09a8f"}
Oct 30 22:07:53.744022 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:53.743998    4871 generic.go:356] "Generic (PLEG): container finished" podID="b28e1c99-b48d-4a4e-a225-ebba779327e4" containerID="c4df035ff8ad9aa06e79ea2ff334fd4a2b3b149452677e5223043972868bed19" exitCode=0
Oct 30 22:07:53.744140 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:53.744070    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h" event={"ID":"b28e1c99-b48d-4a4e-a225-ebba779327e4","Type":"ContainerDied","Data":"c4df035ff8ad9aa06e79ea2ff334fd4a2b3b149452677e5223043972868bed19"}
Oct 30 22:07:54.319527 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.319499    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"
Oct 30 22:07:54.422275 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.422246    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"
Oct 30 22:07:54.426361 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.426335    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"
Oct 30 22:07:54.486260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.485503    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b28e1c99-b48d-4a4e-a225-ebba779327e4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b28e1c99-b48d-4a4e-a225-ebba779327e4-istiod-ca-cert\") pod \"b28e1c99-b48d-4a4e-a225-ebba779327e4\" (UID: \"b28e1c99-b48d-4a4e-a225-ebba779327e4\") "
Oct 30 22:07:54.486260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.485569    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b28e1c99-b48d-4a4e-a225-ebba779327e4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b28e1c99-b48d-4a4e-a225-ebba779327e4-credential-socket\") pod \"b28e1c99-b48d-4a4e-a225-ebba779327e4\" (UID: \"b28e1c99-b48d-4a4e-a225-ebba779327e4\") "
Oct 30 22:07:54.486260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.485606    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b28e1c99-b48d-4a4e-a225-ebba779327e4-kube-api-access-c48rc\" (UniqueName: \"kubernetes.io/projected/b28e1c99-b48d-4a4e-a225-ebba779327e4-kube-api-access-c48rc\") pod \"b28e1c99-b48d-4a4e-a225-ebba779327e4\" (UID: \"b28e1c99-b48d-4a4e-a225-ebba779327e4\") "
Oct 30 22:07:54.486260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.485659    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b28e1c99-b48d-4a4e-a225-ebba779327e4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b28e1c99-b48d-4a4e-a225-ebba779327e4-workload-socket\") pod \"b28e1c99-b48d-4a4e-a225-ebba779327e4\" (UID: \"b28e1c99-b48d-4a4e-a225-ebba779327e4\") "
Oct 30 22:07:54.486260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.485721    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b28e1c99-b48d-4a4e-a225-ebba779327e4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b28e1c99-b48d-4a4e-a225-ebba779327e4-workload-certs\") pod \"b28e1c99-b48d-4a4e-a225-ebba779327e4\" (UID: \"b28e1c99-b48d-4a4e-a225-ebba779327e4\") "
Oct 30 22:07:54.486260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.485755    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b28e1c99-b48d-4a4e-a225-ebba779327e4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b28e1c99-b48d-4a4e-a225-ebba779327e4-istio-envoy\") pod \"b28e1c99-b48d-4a4e-a225-ebba779327e4\" (UID: \"b28e1c99-b48d-4a4e-a225-ebba779327e4\") "
Oct 30 22:07:54.486260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.485942    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b28e1c99-b48d-4a4e-a225-ebba779327e4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b28e1c99-b48d-4a4e-a225-ebba779327e4-istio-podinfo\") pod \"b28e1c99-b48d-4a4e-a225-ebba779327e4\" (UID: \"b28e1c99-b48d-4a4e-a225-ebba779327e4\") "
Oct 30 22:07:54.486260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.485972    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b28e1c99-b48d-4a4e-a225-ebba779327e4-istio-token\" (UniqueName: \"kubernetes.io/projected/b28e1c99-b48d-4a4e-a225-ebba779327e4-istio-token\") pod \"b28e1c99-b48d-4a4e-a225-ebba779327e4\" (UID: \"b28e1c99-b48d-4a4e-a225-ebba779327e4\") "
Oct 30 22:07:54.486260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.486051    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b28e1c99-b48d-4a4e-a225-ebba779327e4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b28e1c99-b48d-4a4e-a225-ebba779327e4-istio-data\") pod \"b28e1c99-b48d-4a4e-a225-ebba779327e4\" (UID: \"b28e1c99-b48d-4a4e-a225-ebba779327e4\") "
Oct 30 22:07:54.486818 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.486766    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b28e1c99-b48d-4a4e-a225-ebba779327e4-workload-socket" pod "b28e1c99-b48d-4a4e-a225-ebba779327e4" (UID: "b28e1c99-b48d-4a4e-a225-ebba779327e4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:07:54.487152 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.487056    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b28e1c99-b48d-4a4e-a225-ebba779327e4-istiod-ca-cert" pod "b28e1c99-b48d-4a4e-a225-ebba779327e4" (UID: "b28e1c99-b48d-4a4e-a225-ebba779327e4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:07:54.487402 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.487368    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b28e1c99-b48d-4a4e-a225-ebba779327e4-credential-socket" pod "b28e1c99-b48d-4a4e-a225-ebba779327e4" (UID: "b28e1c99-b48d-4a4e-a225-ebba779327e4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:07:54.487976 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.487948    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b28e1c99-b48d-4a4e-a225-ebba779327e4-workload-certs" pod "b28e1c99-b48d-4a4e-a225-ebba779327e4" (UID: "b28e1c99-b48d-4a4e-a225-ebba779327e4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:07:54.488361 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.488334    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b28e1c99-b48d-4a4e-a225-ebba779327e4-istio-data" pod "b28e1c99-b48d-4a4e-a225-ebba779327e4" (UID: "b28e1c99-b48d-4a4e-a225-ebba779327e4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:07:54.492275 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.489822    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b28e1c99-b48d-4a4e-a225-ebba779327e4-kube-api-access-c48rc" pod "b28e1c99-b48d-4a4e-a225-ebba779327e4" (UID: "b28e1c99-b48d-4a4e-a225-ebba779327e4"). InnerVolumeSpecName "kube-api-access-c48rc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:07:54.492275 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.490172    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b28e1c99-b48d-4a4e-a225-ebba779327e4-istio-podinfo" pod "b28e1c99-b48d-4a4e-a225-ebba779327e4" (UID: "b28e1c99-b48d-4a4e-a225-ebba779327e4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:07:54.492275 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.490208    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b28e1c99-b48d-4a4e-a225-ebba779327e4-istio-token" pod "b28e1c99-b48d-4a4e-a225-ebba779327e4" (UID: "b28e1c99-b48d-4a4e-a225-ebba779327e4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:07:54.492275 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.490578    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b28e1c99-b48d-4a4e-a225-ebba779327e4-istio-envoy" pod "b28e1c99-b48d-4a4e-a225-ebba779327e4" (UID: "b28e1c99-b48d-4a4e-a225-ebba779327e4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:07:54.588541 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.588437    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ffc32abd-61d0-447a-9050-2ad518ca585f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffc32abd-61d0-447a-9050-2ad518ca585f-credential-socket\") pod \"ffc32abd-61d0-447a-9050-2ad518ca585f\" (UID: \"ffc32abd-61d0-447a-9050-2ad518ca585f\") "
Oct 30 22:07:54.588541 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.588517    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19911666-f25a-4dd7-9789-7b9374b61cfb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19911666-f25a-4dd7-9789-7b9374b61cfb-workload-socket\") pod \"19911666-f25a-4dd7-9789-7b9374b61cfb\" (UID: \"19911666-f25a-4dd7-9789-7b9374b61cfb\") "
Oct 30 22:07:54.588778 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.588565    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/19911666-f25a-4dd7-9789-7b9374b61cfb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19911666-f25a-4dd7-9789-7b9374b61cfb-istiod-ca-cert\") pod \"19911666-f25a-4dd7-9789-7b9374b61cfb\" (UID: \"19911666-f25a-4dd7-9789-7b9374b61cfb\") "
Oct 30 22:07:54.588778 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.588612    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ffc32abd-61d0-447a-9050-2ad518ca585f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffc32abd-61d0-447a-9050-2ad518ca585f-workload-socket\") pod \"ffc32abd-61d0-447a-9050-2ad518ca585f\" (UID: \"ffc32abd-61d0-447a-9050-2ad518ca585f\") "
Oct 30 22:07:54.588778 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.588668    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ffc32abd-61d0-447a-9050-2ad518ca585f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffc32abd-61d0-447a-9050-2ad518ca585f-istio-podinfo\") pod \"ffc32abd-61d0-447a-9050-2ad518ca585f\" (UID: \"ffc32abd-61d0-447a-9050-2ad518ca585f\") "
Oct 30 22:07:54.588778 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.588726    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ffc32abd-61d0-447a-9050-2ad518ca585f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffc32abd-61d0-447a-9050-2ad518ca585f-istio-data\") pod \"ffc32abd-61d0-447a-9050-2ad518ca585f\" (UID: \"ffc32abd-61d0-447a-9050-2ad518ca585f\") "
Oct 30 22:07:54.588778 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.588768    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ffc32abd-61d0-447a-9050-2ad518ca585f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffc32abd-61d0-447a-9050-2ad518ca585f-istiod-ca-cert\") pod \"ffc32abd-61d0-447a-9050-2ad518ca585f\" (UID: \"ffc32abd-61d0-447a-9050-2ad518ca585f\") "
Oct 30 22:07:54.588997 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.588806    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ffc32abd-61d0-447a-9050-2ad518ca585f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffc32abd-61d0-447a-9050-2ad518ca585f-workload-certs\") pod \"ffc32abd-61d0-447a-9050-2ad518ca585f\" (UID: \"ffc32abd-61d0-447a-9050-2ad518ca585f\") "
Oct 30 22:07:54.588997 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.588844    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19911666-f25a-4dd7-9789-7b9374b61cfb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19911666-f25a-4dd7-9789-7b9374b61cfb-istio-envoy\") pod \"19911666-f25a-4dd7-9789-7b9374b61cfb\" (UID: \"19911666-f25a-4dd7-9789-7b9374b61cfb\") "
Oct 30 22:07:54.588997 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.588878    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/19911666-f25a-4dd7-9789-7b9374b61cfb-istio-token\" (UniqueName: \"kubernetes.io/projected/19911666-f25a-4dd7-9789-7b9374b61cfb-istio-token\") pod \"19911666-f25a-4dd7-9789-7b9374b61cfb\" (UID: \"19911666-f25a-4dd7-9789-7b9374b61cfb\") "
Oct 30 22:07:54.588997 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.588938    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19911666-f25a-4dd7-9789-7b9374b61cfb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19911666-f25a-4dd7-9789-7b9374b61cfb-istio-data\") pod \"19911666-f25a-4dd7-9789-7b9374b61cfb\" (UID: \"19911666-f25a-4dd7-9789-7b9374b61cfb\") "
Oct 30 22:07:54.588997 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.588994    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ffc32abd-61d0-447a-9050-2ad518ca585f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffc32abd-61d0-447a-9050-2ad518ca585f-istio-envoy\") pod \"ffc32abd-61d0-447a-9050-2ad518ca585f\" (UID: \"ffc32abd-61d0-447a-9050-2ad518ca585f\") "
Oct 30 22:07:54.589212 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.589060    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ffc32abd-61d0-447a-9050-2ad518ca585f-istio-token\" (UniqueName: \"kubernetes.io/projected/ffc32abd-61d0-447a-9050-2ad518ca585f-istio-token\") pod \"ffc32abd-61d0-447a-9050-2ad518ca585f\" (UID: \"ffc32abd-61d0-447a-9050-2ad518ca585f\") "
Oct 30 22:07:54.589212 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.589095    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/19911666-f25a-4dd7-9789-7b9374b61cfb-kube-api-access-bwqrc\" (UniqueName: \"kubernetes.io/projected/19911666-f25a-4dd7-9789-7b9374b61cfb-kube-api-access-bwqrc\") pod \"19911666-f25a-4dd7-9789-7b9374b61cfb\" (UID: \"19911666-f25a-4dd7-9789-7b9374b61cfb\") "
Oct 30 22:07:54.589212 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.589191    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19911666-f25a-4dd7-9789-7b9374b61cfb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19911666-f25a-4dd7-9789-7b9374b61cfb-workload-certs\") pod \"19911666-f25a-4dd7-9789-7b9374b61cfb\" (UID: \"19911666-f25a-4dd7-9789-7b9374b61cfb\") "
Oct 30 22:07:54.589379 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.589261    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ffc32abd-61d0-447a-9050-2ad518ca585f-kube-api-access-tghmq\" (UniqueName: \"kubernetes.io/projected/ffc32abd-61d0-447a-9050-2ad518ca585f-kube-api-access-tghmq\") pod \"ffc32abd-61d0-447a-9050-2ad518ca585f\" (UID: \"ffc32abd-61d0-447a-9050-2ad518ca585f\") "
Oct 30 22:07:54.589379 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.589294    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/19911666-f25a-4dd7-9789-7b9374b61cfb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19911666-f25a-4dd7-9789-7b9374b61cfb-istio-podinfo\") pod \"19911666-f25a-4dd7-9789-7b9374b61cfb\" (UID: \"19911666-f25a-4dd7-9789-7b9374b61cfb\") "
Oct 30 22:07:54.589379 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.589335    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19911666-f25a-4dd7-9789-7b9374b61cfb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19911666-f25a-4dd7-9789-7b9374b61cfb-credential-socket\") pod \"19911666-f25a-4dd7-9789-7b9374b61cfb\" (UID: \"19911666-f25a-4dd7-9789-7b9374b61cfb\") "
Oct 30 22:07:54.589782 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.589753    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19911666-f25a-4dd7-9789-7b9374b61cfb-credential-socket" pod "19911666-f25a-4dd7-9789-7b9374b61cfb" (UID: "19911666-f25a-4dd7-9789-7b9374b61cfb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:07:54.590509 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.590186    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19911666-f25a-4dd7-9789-7b9374b61cfb-workload-socket" pod "19911666-f25a-4dd7-9789-7b9374b61cfb" (UID: "19911666-f25a-4dd7-9789-7b9374b61cfb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:07:54.590509 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.590405    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19911666-f25a-4dd7-9789-7b9374b61cfb-istio-data" pod "19911666-f25a-4dd7-9789-7b9374b61cfb" (UID: "19911666-f25a-4dd7-9789-7b9374b61cfb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:07:54.590861 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.590608    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b28e1c99-b48d-4a4e-a225-ebba779327e4-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:54.590861 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.590645    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b28e1c99-b48d-4a4e-a225-ebba779327e4-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:54.590861 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.590663    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19911666-f25a-4dd7-9789-7b9374b61cfb-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:54.590861 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.590680    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b28e1c99-b48d-4a4e-a225-ebba779327e4-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:54.590861 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.590698    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19911666-f25a-4dd7-9789-7b9374b61cfb-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:54.590861 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.590720    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b28e1c99-b48d-4a4e-a225-ebba779327e4-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:54.590861 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.590734    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b28e1c99-b48d-4a4e-a225-ebba779327e4-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:54.590861 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.590748    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c48rc\" (UniqueName: \"kubernetes.io/projected/b28e1c99-b48d-4a4e-a225-ebba779327e4-kube-api-access-c48rc\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:54.590861 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.590767    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b28e1c99-b48d-4a4e-a225-ebba779327e4-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:54.590861 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.590787    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b28e1c99-b48d-4a4e-a225-ebba779327e4-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:54.590861 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.590800    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b28e1c99-b48d-4a4e-a225-ebba779327e4-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:54.590861 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.590662    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19911666-f25a-4dd7-9789-7b9374b61cfb-workload-certs" pod "19911666-f25a-4dd7-9789-7b9374b61cfb" (UID: "19911666-f25a-4dd7-9789-7b9374b61cfb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:07:54.590861 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.590738    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19911666-f25a-4dd7-9789-7b9374b61cfb-istiod-ca-cert" pod "19911666-f25a-4dd7-9789-7b9374b61cfb" (UID: "19911666-f25a-4dd7-9789-7b9374b61cfb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:07:54.591611 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.591381    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffc32abd-61d0-447a-9050-2ad518ca585f-istio-data" pod "ffc32abd-61d0-447a-9050-2ad518ca585f" (UID: "ffc32abd-61d0-447a-9050-2ad518ca585f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:07:54.591611 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.591541    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffc32abd-61d0-447a-9050-2ad518ca585f-credential-socket" pod "ffc32abd-61d0-447a-9050-2ad518ca585f" (UID: "ffc32abd-61d0-447a-9050-2ad518ca585f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:07:54.591799 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.591741    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffc32abd-61d0-447a-9050-2ad518ca585f-istiod-ca-cert" pod "ffc32abd-61d0-447a-9050-2ad518ca585f" (UID: "ffc32abd-61d0-447a-9050-2ad518ca585f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:07:54.592367 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.591923    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffc32abd-61d0-447a-9050-2ad518ca585f-workload-socket" pod "ffc32abd-61d0-447a-9050-2ad518ca585f" (UID: "ffc32abd-61d0-447a-9050-2ad518ca585f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:07:54.593090 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.592932    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19911666-f25a-4dd7-9789-7b9374b61cfb-kube-api-access-bwqrc" pod "19911666-f25a-4dd7-9789-7b9374b61cfb" (UID: "19911666-f25a-4dd7-9789-7b9374b61cfb"). InnerVolumeSpecName "kube-api-access-bwqrc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:07:54.593090 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.593004    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19911666-f25a-4dd7-9789-7b9374b61cfb-istio-token" pod "19911666-f25a-4dd7-9789-7b9374b61cfb" (UID: "19911666-f25a-4dd7-9789-7b9374b61cfb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:07:54.593090 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.593012    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19911666-f25a-4dd7-9789-7b9374b61cfb-istio-envoy" pod "19911666-f25a-4dd7-9789-7b9374b61cfb" (UID: "19911666-f25a-4dd7-9789-7b9374b61cfb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:07:54.593522 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.593499    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffc32abd-61d0-447a-9050-2ad518ca585f-workload-certs" pod "ffc32abd-61d0-447a-9050-2ad518ca585f" (UID: "ffc32abd-61d0-447a-9050-2ad518ca585f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:07:54.594589 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.594540    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/19911666-f25a-4dd7-9789-7b9374b61cfb-istio-podinfo" pod "19911666-f25a-4dd7-9789-7b9374b61cfb" (UID: "19911666-f25a-4dd7-9789-7b9374b61cfb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:07:54.594689 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.594595    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffc32abd-61d0-447a-9050-2ad518ca585f-istio-token" pod "ffc32abd-61d0-447a-9050-2ad518ca585f" (UID: "ffc32abd-61d0-447a-9050-2ad518ca585f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:07:54.595340 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.595316    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ffc32abd-61d0-447a-9050-2ad518ca585f-istio-podinfo" pod "ffc32abd-61d0-447a-9050-2ad518ca585f" (UID: "ffc32abd-61d0-447a-9050-2ad518ca585f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:07:54.595648 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.595628    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffc32abd-61d0-447a-9050-2ad518ca585f-istio-envoy" pod "ffc32abd-61d0-447a-9050-2ad518ca585f" (UID: "ffc32abd-61d0-447a-9050-2ad518ca585f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:07:54.608231 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.608179    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffc32abd-61d0-447a-9050-2ad518ca585f-kube-api-access-tghmq" pod "ffc32abd-61d0-447a-9050-2ad518ca585f" (UID: "ffc32abd-61d0-447a-9050-2ad518ca585f"). InnerVolumeSpecName "kube-api-access-tghmq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:07:54.692356 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.692288    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19911666-f25a-4dd7-9789-7b9374b61cfb-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:54.692356 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.692322    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffc32abd-61d0-447a-9050-2ad518ca585f-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:54.692356 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.692332    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffc32abd-61d0-447a-9050-2ad518ca585f-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:54.692356 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.692341    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffc32abd-61d0-447a-9050-2ad518ca585f-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:54.692356 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.692350    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffc32abd-61d0-447a-9050-2ad518ca585f-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:54.692356 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.692359    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffc32abd-61d0-447a-9050-2ad518ca585f-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:54.692356 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.692370    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19911666-f25a-4dd7-9789-7b9374b61cfb-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:54.692710 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.692379    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19911666-f25a-4dd7-9789-7b9374b61cfb-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:54.692710 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.692388    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19911666-f25a-4dd7-9789-7b9374b61cfb-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:54.692710 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.692396    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffc32abd-61d0-447a-9050-2ad518ca585f-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:54.692710 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.692404    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffc32abd-61d0-447a-9050-2ad518ca585f-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:54.692710 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.692412    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bwqrc\" (UniqueName: \"kubernetes.io/projected/19911666-f25a-4dd7-9789-7b9374b61cfb-kube-api-access-bwqrc\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:54.692710 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.692423    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19911666-f25a-4dd7-9789-7b9374b61cfb-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:54.692710 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.692439    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tghmq\" (UniqueName: \"kubernetes.io/projected/ffc32abd-61d0-447a-9050-2ad518ca585f-kube-api-access-tghmq\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:54.692710 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.692447    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19911666-f25a-4dd7-9789-7b9374b61cfb-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:54.692710 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.692456    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffc32abd-61d0-447a-9050-2ad518ca585f-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:07:54.754147 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.754104    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j" event={"ID":"19911666-f25a-4dd7-9789-7b9374b61cfb","Type":"ContainerDied","Data":"00c42bd448198dcc73803fdf7d05aff173eff79290426e5a67cf89f2ed3510c9"}
Oct 30 22:07:54.754381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.754110    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"
Oct 30 22:07:54.754381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.754217    4871 scope.go:122] "RemoveContainer" containerID="af426aeb828c2148a3ba12ce18f7495dc345415ee0419868e346a97bb8a09a8f"
Oct 30 22:07:54.755996 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.755966    4871 generic.go:356] "Generic (PLEG): container finished" podID="ffc32abd-61d0-447a-9050-2ad518ca585f" containerID="74b1a584476cfaea2631fc7f42dd12b0cf532676d1098a88eb9a215d8bb7905b" exitCode=0
Oct 30 22:07:54.756145 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.755968    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8" event={"ID":"ffc32abd-61d0-447a-9050-2ad518ca585f","Type":"ContainerDied","Data":"74b1a584476cfaea2631fc7f42dd12b0cf532676d1098a88eb9a215d8bb7905b"}
Oct 30 22:07:54.756145 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.756057    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8" event={"ID":"ffc32abd-61d0-447a-9050-2ad518ca585f","Type":"ContainerDied","Data":"42c90767c3085aacba721ec56da1b4ac3cd5434b541c38c1a1a539c96d2000b2"}
Oct 30 22:07:54.756145 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.756005    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"
Oct 30 22:07:54.758143 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.758117    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h" event={"ID":"b28e1c99-b48d-4a4e-a225-ebba779327e4","Type":"ContainerDied","Data":"98ac329ac724c42b57b6dac2f63e075039febe24ca20e5464552d3e08377d68b"}
Oct 30 22:07:54.758423 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.758226    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"
Oct 30 22:07:54.788365 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.788052    4871 scope.go:122] "RemoveContainer" containerID="74b1a584476cfaea2631fc7f42dd12b0cf532676d1098a88eb9a215d8bb7905b"
Oct 30 22:07:54.814641 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.814574    4871 scope.go:122] "RemoveContainer" containerID="74b1a584476cfaea2631fc7f42dd12b0cf532676d1098a88eb9a215d8bb7905b"
Oct 30 22:07:54.814964 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:07:54.814940    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74b1a584476cfaea2631fc7f42dd12b0cf532676d1098a88eb9a215d8bb7905b\": container with ID starting with 74b1a584476cfaea2631fc7f42dd12b0cf532676d1098a88eb9a215d8bb7905b not found: ID does not exist" containerID="74b1a584476cfaea2631fc7f42dd12b0cf532676d1098a88eb9a215d8bb7905b"
Oct 30 22:07:54.815070 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.814978    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74b1a584476cfaea2631fc7f42dd12b0cf532676d1098a88eb9a215d8bb7905b"} err="failed to get container status \"74b1a584476cfaea2631fc7f42dd12b0cf532676d1098a88eb9a215d8bb7905b\": rpc error: code = NotFound desc = could not find container \"74b1a584476cfaea2631fc7f42dd12b0cf532676d1098a88eb9a215d8bb7905b\": container with ID starting with 74b1a584476cfaea2631fc7f42dd12b0cf532676d1098a88eb9a215d8bb7905b not found: ID does not exist"
Oct 30 22:07:54.815070 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.815003    4871 scope.go:122] "RemoveContainer" containerID="c4df035ff8ad9aa06e79ea2ff334fd4a2b3b149452677e5223043972868bed19"
Oct 30 22:07:54.967789 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.967755    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"]
Oct 30 22:07:54.972140 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:54.972111    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-bpm9h"]
Oct 30 22:07:55.005389 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:55.005351    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:55.096261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:55.096050    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"]
Oct 30 22:07:55.096467 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:55.096320    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mrq5j"]
Oct 30 22:07:55.198082 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:55.198046    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"]
Oct 30 22:07:55.207486 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:55.207452    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-btfp8"]
Oct 30 22:07:55.961862 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:55.961822    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="19911666-f25a-4dd7-9789-7b9374b61cfb" path="/var/lib/kubelet/pods/19911666-f25a-4dd7-9789-7b9374b61cfb/volumes"
Oct 30 22:07:55.962507 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:55.962487    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b28e1c99-b48d-4a4e-a225-ebba779327e4" path="/var/lib/kubelet/pods/b28e1c99-b48d-4a4e-a225-ebba779327e4/volumes"
Oct 30 22:07:55.962941 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:55.962927    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ffc32abd-61d0-447a-9050-2ad518ca585f" path="/var/lib/kubelet/pods/ffc32abd-61d0-447a-9050-2ad518ca585f/volumes"
Oct 30 22:07:56.811787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:56.811749    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:56.977016 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:56.976984    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:57.169844 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:57.169805    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"]
Oct 30 22:07:57.170092 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:57.170067    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr" podUID="ef03dee1-e693-42a0-ace8-4a28e099b92b" containerName="istio-proxy" containerID="cri-o://0dfe7f5862ac6b26a9e1bed74b43224d640b6f548b514e15e5eb1b2ad8e7746d" gracePeriod=30
Oct 30 22:07:57.175087 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:07:57.175055    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:00.050436 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:00.050396    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:00.526426 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:00.526387    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"]
Oct 30 22:08:00.526678 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:00.526652    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt" podUID="52b9b4dc-149f-4650-81f7-7f0234d9c6d8" containerName="istio-proxy" containerID="cri-o://cd2786aec1e8db07dcd8ecd2304de1e7078e3f076a7fd2cc0b95628e9f179756" gracePeriod=30
Oct 30 22:08:02.469559 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:02.469531    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"
Oct 30 22:08:02.540837 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:02.540793    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ef03dee1-e693-42a0-ace8-4a28e099b92b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef03dee1-e693-42a0-ace8-4a28e099b92b-istio-podinfo\") pod \"ef03dee1-e693-42a0-ace8-4a28e099b92b\" (UID: \"ef03dee1-e693-42a0-ace8-4a28e099b92b\") "
Oct 30 22:08:02.541042 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:02.540876    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef03dee1-e693-42a0-ace8-4a28e099b92b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef03dee1-e693-42a0-ace8-4a28e099b92b-istio-data\") pod \"ef03dee1-e693-42a0-ace8-4a28e099b92b\" (UID: \"ef03dee1-e693-42a0-ace8-4a28e099b92b\") "
Oct 30 22:08:02.541042 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:02.540972    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ef03dee1-e693-42a0-ace8-4a28e099b92b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef03dee1-e693-42a0-ace8-4a28e099b92b-istiod-ca-cert\") pod \"ef03dee1-e693-42a0-ace8-4a28e099b92b\" (UID: \"ef03dee1-e693-42a0-ace8-4a28e099b92b\") "
Oct 30 22:08:02.541181 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:02.541100    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef03dee1-e693-42a0-ace8-4a28e099b92b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef03dee1-e693-42a0-ace8-4a28e099b92b-workload-certs\") pod \"ef03dee1-e693-42a0-ace8-4a28e099b92b\" (UID: \"ef03dee1-e693-42a0-ace8-4a28e099b92b\") "
Oct 30 22:08:02.541227 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:02.541179    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef03dee1-e693-42a0-ace8-4a28e099b92b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef03dee1-e693-42a0-ace8-4a28e099b92b-workload-socket\") pod \"ef03dee1-e693-42a0-ace8-4a28e099b92b\" (UID: \"ef03dee1-e693-42a0-ace8-4a28e099b92b\") "
Oct 30 22:08:02.541343 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:02.541227    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef03dee1-e693-42a0-ace8-4a28e099b92b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef03dee1-e693-42a0-ace8-4a28e099b92b-istio-envoy\") pod \"ef03dee1-e693-42a0-ace8-4a28e099b92b\" (UID: \"ef03dee1-e693-42a0-ace8-4a28e099b92b\") "
Oct 30 22:08:02.541343 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:02.541283    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef03dee1-e693-42a0-ace8-4a28e099b92b-istio-data" pod "ef03dee1-e693-42a0-ace8-4a28e099b92b" (UID: "ef03dee1-e693-42a0-ace8-4a28e099b92b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:02.541460 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:02.541340    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef03dee1-e693-42a0-ace8-4a28e099b92b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef03dee1-e693-42a0-ace8-4a28e099b92b-credential-socket\") pod \"ef03dee1-e693-42a0-ace8-4a28e099b92b\" (UID: \"ef03dee1-e693-42a0-ace8-4a28e099b92b\") "
Oct 30 22:08:02.541460 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:02.541387    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef03dee1-e693-42a0-ace8-4a28e099b92b-workload-certs" pod "ef03dee1-e693-42a0-ace8-4a28e099b92b" (UID: "ef03dee1-e693-42a0-ace8-4a28e099b92b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:02.541460 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:02.541407    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ef03dee1-e693-42a0-ace8-4a28e099b92b-istio-token\" (UniqueName: \"kubernetes.io/projected/ef03dee1-e693-42a0-ace8-4a28e099b92b-istio-token\") pod \"ef03dee1-e693-42a0-ace8-4a28e099b92b\" (UID: \"ef03dee1-e693-42a0-ace8-4a28e099b92b\") "
Oct 30 22:08:02.541622 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:02.541443    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef03dee1-e693-42a0-ace8-4a28e099b92b-istiod-ca-cert" pod "ef03dee1-e693-42a0-ace8-4a28e099b92b" (UID: "ef03dee1-e693-42a0-ace8-4a28e099b92b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:08:02.541622 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:02.541456    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ef03dee1-e693-42a0-ace8-4a28e099b92b-kube-api-access-7gkvb\" (UniqueName: \"kubernetes.io/projected/ef03dee1-e693-42a0-ace8-4a28e099b92b-kube-api-access-7gkvb\") pod \"ef03dee1-e693-42a0-ace8-4a28e099b92b\" (UID: \"ef03dee1-e693-42a0-ace8-4a28e099b92b\") "
Oct 30 22:08:02.541622 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:02.541584    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef03dee1-e693-42a0-ace8-4a28e099b92b-credential-socket" pod "ef03dee1-e693-42a0-ace8-4a28e099b92b" (UID: "ef03dee1-e693-42a0-ace8-4a28e099b92b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:02.541622 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:02.541578    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef03dee1-e693-42a0-ace8-4a28e099b92b-workload-socket" pod "ef03dee1-e693-42a0-ace8-4a28e099b92b" (UID: "ef03dee1-e693-42a0-ace8-4a28e099b92b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:02.542307 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:02.542278    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef03dee1-e693-42a0-ace8-4a28e099b92b-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:08:02.542432 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:02.542311    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef03dee1-e693-42a0-ace8-4a28e099b92b-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:08:02.542432 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:02.542326    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef03dee1-e693-42a0-ace8-4a28e099b92b-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:08:02.542432 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:02.542343    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef03dee1-e693-42a0-ace8-4a28e099b92b-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:08:02.542432 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:02.542359    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef03dee1-e693-42a0-ace8-4a28e099b92b-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:08:02.543331 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:02.543301    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ef03dee1-e693-42a0-ace8-4a28e099b92b-istio-podinfo" pod "ef03dee1-e693-42a0-ace8-4a28e099b92b" (UID: "ef03dee1-e693-42a0-ace8-4a28e099b92b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:08:02.543546 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:02.543528    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef03dee1-e693-42a0-ace8-4a28e099b92b-istio-envoy" pod "ef03dee1-e693-42a0-ace8-4a28e099b92b" (UID: "ef03dee1-e693-42a0-ace8-4a28e099b92b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:02.543943 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:02.543926    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef03dee1-e693-42a0-ace8-4a28e099b92b-istio-token" pod "ef03dee1-e693-42a0-ace8-4a28e099b92b" (UID: "ef03dee1-e693-42a0-ace8-4a28e099b92b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:08:02.544025 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:02.543952    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef03dee1-e693-42a0-ace8-4a28e099b92b-kube-api-access-7gkvb" pod "ef03dee1-e693-42a0-ace8-4a28e099b92b" (UID: "ef03dee1-e693-42a0-ace8-4a28e099b92b"). InnerVolumeSpecName "kube-api-access-7gkvb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:08:02.644280 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:02.644200    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef03dee1-e693-42a0-ace8-4a28e099b92b-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:08:02.644280 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:02.644256    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef03dee1-e693-42a0-ace8-4a28e099b92b-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:08:02.644280 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:02.644274    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7gkvb\" (UniqueName: \"kubernetes.io/projected/ef03dee1-e693-42a0-ace8-4a28e099b92b-kube-api-access-7gkvb\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:08:02.644280 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:02.644287    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef03dee1-e693-42a0-ace8-4a28e099b92b-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:08:02.835735 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:02.835627    4871 generic.go:356] "Generic (PLEG): container finished" podID="ef03dee1-e693-42a0-ace8-4a28e099b92b" containerID="0dfe7f5862ac6b26a9e1bed74b43224d640b6f548b514e15e5eb1b2ad8e7746d" exitCode=0
Oct 30 22:08:02.835735 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:02.835716    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr" event={"ID":"ef03dee1-e693-42a0-ace8-4a28e099b92b","Type":"ContainerDied","Data":"0dfe7f5862ac6b26a9e1bed74b43224d640b6f548b514e15e5eb1b2ad8e7746d"}
Oct 30 22:08:02.835986 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:02.835770    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr" event={"ID":"ef03dee1-e693-42a0-ace8-4a28e099b92b","Type":"ContainerDied","Data":"0897c2a8d63467761ecd998f506bdc8eb3c4108531eabb83d21a4bf13cf743eb"}
Oct 30 22:08:02.835986 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:02.835802    4871 scope.go:122] "RemoveContainer" containerID="0dfe7f5862ac6b26a9e1bed74b43224d640b6f548b514e15e5eb1b2ad8e7746d"
Oct 30 22:08:02.835986 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:02.835940    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"
Oct 30 22:08:02.854682 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:02.854655    4871 scope.go:122] "RemoveContainer" containerID="2b49035c6e6dbaec7dd1491af63a36a98ff5b497551fe9664f34702e21c712c9"
Oct 30 22:08:02.866406 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:02.866372    4871 scope.go:122] "RemoveContainer" containerID="0dfe7f5862ac6b26a9e1bed74b43224d640b6f548b514e15e5eb1b2ad8e7746d"
Oct 30 22:08:02.866769 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:08:02.866746    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0dfe7f5862ac6b26a9e1bed74b43224d640b6f548b514e15e5eb1b2ad8e7746d\": container with ID starting with 0dfe7f5862ac6b26a9e1bed74b43224d640b6f548b514e15e5eb1b2ad8e7746d not found: ID does not exist" containerID="0dfe7f5862ac6b26a9e1bed74b43224d640b6f548b514e15e5eb1b2ad8e7746d"
Oct 30 22:08:02.866845 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:02.866781    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0dfe7f5862ac6b26a9e1bed74b43224d640b6f548b514e15e5eb1b2ad8e7746d"} err="failed to get container status \"0dfe7f5862ac6b26a9e1bed74b43224d640b6f548b514e15e5eb1b2ad8e7746d\": rpc error: code = NotFound desc = could not find container \"0dfe7f5862ac6b26a9e1bed74b43224d640b6f548b514e15e5eb1b2ad8e7746d\": container with ID starting with 0dfe7f5862ac6b26a9e1bed74b43224d640b6f548b514e15e5eb1b2ad8e7746d not found: ID does not exist"
Oct 30 22:08:02.866845 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:02.866801    4871 scope.go:122] "RemoveContainer" containerID="2b49035c6e6dbaec7dd1491af63a36a98ff5b497551fe9664f34702e21c712c9"
Oct 30 22:08:02.867137 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:08:02.867118    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b49035c6e6dbaec7dd1491af63a36a98ff5b497551fe9664f34702e21c712c9\": container with ID starting with 2b49035c6e6dbaec7dd1491af63a36a98ff5b497551fe9664f34702e21c712c9 not found: ID does not exist" containerID="2b49035c6e6dbaec7dd1491af63a36a98ff5b497551fe9664f34702e21c712c9"
Oct 30 22:08:02.867201 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:02.867141    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b49035c6e6dbaec7dd1491af63a36a98ff5b497551fe9664f34702e21c712c9"} err="failed to get container status \"2b49035c6e6dbaec7dd1491af63a36a98ff5b497551fe9664f34702e21c712c9\": rpc error: code = NotFound desc = could not find container \"2b49035c6e6dbaec7dd1491af63a36a98ff5b497551fe9664f34702e21c712c9\": container with ID starting with 2b49035c6e6dbaec7dd1491af63a36a98ff5b497551fe9664f34702e21c712c9 not found: ID does not exist"
Oct 30 22:08:02.996341 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:02.996302    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"]
Oct 30 22:08:03.010578 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:03.010536    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-2d2f63-openshift-default-786db8c96c-q8tjr"]
Oct 30 22:08:03.961773 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:03.961732    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ef03dee1-e693-42a0-ace8-4a28e099b92b" path="/var/lib/kubelet/pods/ef03dee1-e693-42a0-ace8-4a28e099b92b/volumes"
Oct 30 22:08:05.118448 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:05.118402    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:05.570671 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:08:05.570629    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:08:05.821840 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:05.821811    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"
Oct 30 22:08:05.864427 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:05.864392    4871 generic.go:356] "Generic (PLEG): container finished" podID="52b9b4dc-149f-4650-81f7-7f0234d9c6d8" containerID="cd2786aec1e8db07dcd8ecd2304de1e7078e3f076a7fd2cc0b95628e9f179756" exitCode=0
Oct 30 22:08:05.864600 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:05.864446    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"
Oct 30 22:08:05.864600 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:05.864443    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt" event={"ID":"52b9b4dc-149f-4650-81f7-7f0234d9c6d8","Type":"ContainerDied","Data":"cd2786aec1e8db07dcd8ecd2304de1e7078e3f076a7fd2cc0b95628e9f179756"}
Oct 30 22:08:05.864600 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:05.864580    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt" event={"ID":"52b9b4dc-149f-4650-81f7-7f0234d9c6d8","Type":"ContainerDied","Data":"a14292423c58d043d91759649fc807d8fd8cef4e136d3b47bf0869216aa98f4b"}
Oct 30 22:08:05.864732 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:05.864608    4871 scope.go:122] "RemoveContainer" containerID="cd2786aec1e8db07dcd8ecd2304de1e7078e3f076a7fd2cc0b95628e9f179756"
Oct 30 22:08:05.876144 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:05.876119    4871 scope.go:122] "RemoveContainer" containerID="cd2786aec1e8db07dcd8ecd2304de1e7078e3f076a7fd2cc0b95628e9f179756"
Oct 30 22:08:05.876476 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:08:05.876456    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd2786aec1e8db07dcd8ecd2304de1e7078e3f076a7fd2cc0b95628e9f179756\": container with ID starting with cd2786aec1e8db07dcd8ecd2304de1e7078e3f076a7fd2cc0b95628e9f179756 not found: ID does not exist" containerID="cd2786aec1e8db07dcd8ecd2304de1e7078e3f076a7fd2cc0b95628e9f179756"
Oct 30 22:08:05.876541 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:05.876492    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd2786aec1e8db07dcd8ecd2304de1e7078e3f076a7fd2cc0b95628e9f179756"} err="failed to get container status \"cd2786aec1e8db07dcd8ecd2304de1e7078e3f076a7fd2cc0b95628e9f179756\": rpc error: code = NotFound desc = could not find container \"cd2786aec1e8db07dcd8ecd2304de1e7078e3f076a7fd2cc0b95628e9f179756\": container with ID starting with cd2786aec1e8db07dcd8ecd2304de1e7078e3f076a7fd2cc0b95628e9f179756 not found: ID does not exist"
Oct 30 22:08:05.914151 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:05.914113    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-istiod-ca-cert\") pod \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\" (UID: \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\") "
Oct 30 22:08:05.914364 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:05.914186    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-credential-socket\") pod \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\" (UID: \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\") "
Oct 30 22:08:05.914415 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:05.914371    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-istio-podinfo\") pod \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\" (UID: \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\") "
Oct 30 22:08:05.914461 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:05.914419    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-istio-data\") pod \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\" (UID: \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\") "
Oct 30 22:08:05.914520 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:05.914460    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-istio-token\" (UniqueName: \"kubernetes.io/projected/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-istio-token\") pod \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\" (UID: \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\") "
Oct 30 22:08:05.914520 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:05.914484    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-credential-socket" pod "52b9b4dc-149f-4650-81f7-7f0234d9c6d8" (UID: "52b9b4dc-149f-4650-81f7-7f0234d9c6d8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:05.914645 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:05.914517    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-istio-envoy\") pod \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\" (UID: \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\") "
Oct 30 22:08:05.914645 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:05.914547    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-istiod-ca-cert" pod "52b9b4dc-149f-4650-81f7-7f0234d9c6d8" (UID: "52b9b4dc-149f-4650-81f7-7f0234d9c6d8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:08:05.914645 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:05.914564    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-workload-certs\") pod \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\" (UID: \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\") "
Oct 30 22:08:05.914797 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:05.914648    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-workload-socket\") pod \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\" (UID: \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\") "
Oct 30 22:08:05.914857 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:05.914800    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-kube-api-access-2x2z6\" (UniqueName: \"kubernetes.io/projected/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-kube-api-access-2x2z6\") pod \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\" (UID: \"52b9b4dc-149f-4650-81f7-7f0234d9c6d8\") "
Oct 30 22:08:05.914916 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:05.914868    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-istio-data" pod "52b9b4dc-149f-4650-81f7-7f0234d9c6d8" (UID: "52b9b4dc-149f-4650-81f7-7f0234d9c6d8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:05.914916 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:05.914891    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-workload-socket" pod "52b9b4dc-149f-4650-81f7-7f0234d9c6d8" (UID: "52b9b4dc-149f-4650-81f7-7f0234d9c6d8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:05.915191 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:05.915136    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-workload-certs" pod "52b9b4dc-149f-4650-81f7-7f0234d9c6d8" (UID: "52b9b4dc-149f-4650-81f7-7f0234d9c6d8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:05.915450 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:05.915427    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:08:05.915543 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:05.915454    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:08:05.915543 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:05.915470    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:08:05.915543 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:05.915486    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:08:05.915543 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:05.915500    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:08:05.916836 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:05.916814    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-istio-envoy" pod "52b9b4dc-149f-4650-81f7-7f0234d9c6d8" (UID: "52b9b4dc-149f-4650-81f7-7f0234d9c6d8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:05.916944 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:05.916927    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-kube-api-access-2x2z6" pod "52b9b4dc-149f-4650-81f7-7f0234d9c6d8" (UID: "52b9b4dc-149f-4650-81f7-7f0234d9c6d8"). InnerVolumeSpecName "kube-api-access-2x2z6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:08:05.917001 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:05.916955    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-istio-token" pod "52b9b4dc-149f-4650-81f7-7f0234d9c6d8" (UID: "52b9b4dc-149f-4650-81f7-7f0234d9c6d8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:08:05.917042 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:05.917023    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-istio-podinfo" pod "52b9b4dc-149f-4650-81f7-7f0234d9c6d8" (UID: "52b9b4dc-149f-4650-81f7-7f0234d9c6d8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:08:06.017602 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:06.017559    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2x2z6\" (UniqueName: \"kubernetes.io/projected/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-kube-api-access-2x2z6\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:08:06.017602 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:06.017604    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:08:06.017797 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:06.017620    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:08:06.017797 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:06.017635    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52b9b4dc-149f-4650-81f7-7f0234d9c6d8-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:08:06.278964 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:06.278929    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"]
Oct 30 22:08:06.282946 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:06.282912    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-djcgt"]
Oct 30 22:08:07.960605 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:07.960564    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="52b9b4dc-149f-4650-81f7-7f0234d9c6d8" path="/var/lib/kubelet/pods/52b9b4dc-149f-4650-81f7-7f0234d9c6d8/volumes"
Oct 30 22:08:09.627269 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:09.627215    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:09.860918 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:09.860883    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:10.015817 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:10.015719    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:10.671966 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:10.671936    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:12.371828 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:12.371791    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:12.530374 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:12.530337    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:14.436843 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:14.436799    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:16.403179 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:16.403144    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:18.804845 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:18.804770    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4c260505-796c8fbff9-tdghh"]
Oct 30 22:08:18.805393 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:18.805066    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4c260505-796c8fbff9-tdghh" podUID="80d35579-62f6-47a6-bcab-37f16bec576a" containerName="maas-api" containerID="cri-o://ed934a3670d468e7b62c3e587187b2ed25a72b0b8d8d57519a18ab4e93addd80" gracePeriod=30
Oct 30 22:08:18.958139 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:18.958095    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:18.981820 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:18.981788    4871 generic.go:356] "Generic (PLEG): container finished" podID="80d35579-62f6-47a6-bcab-37f16bec576a" containerID="ed934a3670d468e7b62c3e587187b2ed25a72b0b8d8d57519a18ab4e93addd80" exitCode=0
Oct 30 22:08:18.981955 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:18.981828    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4c260505-796c8fbff9-tdghh" event={"ID":"80d35579-62f6-47a6-bcab-37f16bec576a","Type":"ContainerDied","Data":"ed934a3670d468e7b62c3e587187b2ed25a72b0b8d8d57519a18ab4e93addd80"}
Oct 30 22:08:19.151312 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:19.151282    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4c260505-796c8fbff9-tdghh"
Oct 30 22:08:19.286264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:19.285531    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/80d35579-62f6-47a6-bcab-37f16bec576a-kube-api-access-ctg8k\" (UniqueName: \"kubernetes.io/projected/80d35579-62f6-47a6-bcab-37f16bec576a-kube-api-access-ctg8k\") pod \"80d35579-62f6-47a6-bcab-37f16bec576a\" (UID: \"80d35579-62f6-47a6-bcab-37f16bec576a\") "
Oct 30 22:08:19.286264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:19.285720    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/80d35579-62f6-47a6-bcab-37f16bec576a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/80d35579-62f6-47a6-bcab-37f16bec576a-metrics-certs\") pod \"80d35579-62f6-47a6-bcab-37f16bec576a\" (UID: \"80d35579-62f6-47a6-bcab-37f16bec576a\") "
Oct 30 22:08:19.286264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:19.285805    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/80d35579-62f6-47a6-bcab-37f16bec576a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80d35579-62f6-47a6-bcab-37f16bec576a-openshift-service-ca\") pod \"80d35579-62f6-47a6-bcab-37f16bec576a\" (UID: \"80d35579-62f6-47a6-bcab-37f16bec576a\") "
Oct 30 22:08:19.286264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:19.285881    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/80d35579-62f6-47a6-bcab-37f16bec576a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80d35579-62f6-47a6-bcab-37f16bec576a-maas-api-tls\") pod \"80d35579-62f6-47a6-bcab-37f16bec576a\" (UID: \"80d35579-62f6-47a6-bcab-37f16bec576a\") "
Oct 30 22:08:19.299731 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:19.299687    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80d35579-62f6-47a6-bcab-37f16bec576a-maas-api-tls" pod "80d35579-62f6-47a6-bcab-37f16bec576a" (UID: "80d35579-62f6-47a6-bcab-37f16bec576a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:08:19.300015 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:19.299986    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80d35579-62f6-47a6-bcab-37f16bec576a-openshift-service-ca" pod "80d35579-62f6-47a6-bcab-37f16bec576a" (UID: "80d35579-62f6-47a6-bcab-37f16bec576a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:08:19.357470 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:19.353096    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80d35579-62f6-47a6-bcab-37f16bec576a-kube-api-access-ctg8k" pod "80d35579-62f6-47a6-bcab-37f16bec576a" (UID: "80d35579-62f6-47a6-bcab-37f16bec576a"). InnerVolumeSpecName "kube-api-access-ctg8k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:08:19.358972 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:19.358911    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80d35579-62f6-47a6-bcab-37f16bec576a-metrics-certs" pod "80d35579-62f6-47a6-bcab-37f16bec576a" (UID: "80d35579-62f6-47a6-bcab-37f16bec576a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:08:19.388780 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:19.388685    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ctg8k\" (UniqueName: \"kubernetes.io/projected/80d35579-62f6-47a6-bcab-37f16bec576a-kube-api-access-ctg8k\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:08:19.388780 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:19.388720    4871 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80d35579-62f6-47a6-bcab-37f16bec576a-metrics-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:08:19.388780 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:19.388729    4871 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80d35579-62f6-47a6-bcab-37f16bec576a-openshift-service-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:08:19.388780 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:19.388739    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80d35579-62f6-47a6-bcab-37f16bec576a-maas-api-tls\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:08:20.045047 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:20.044400    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4c260505-796c8fbff9-tdghh"
Oct 30 22:08:20.045047 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:20.044406    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4c260505-796c8fbff9-tdghh" event={"ID":"80d35579-62f6-47a6-bcab-37f16bec576a","Type":"ContainerDied","Data":"63dc017c9a0f9dd911f93f118ef5c955e367730937db06a3175234d07ad078f0"}
Oct 30 22:08:20.045047 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:20.044540    4871 scope.go:122] "RemoveContainer" containerID="ed934a3670d468e7b62c3e587187b2ed25a72b0b8d8d57519a18ab4e93addd80"
Oct 30 22:08:20.203697 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:20.203659    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4c260505-796c8fbff9-tdghh"]
Oct 30 22:08:20.250172 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:20.250123    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4c260505-796c8fbff9-tdghh"]
Oct 30 22:08:20.588403 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:20.588316    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:20.848641 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:20.848558    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:21.010585 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:21.010547    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:21.554787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:21.554748    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:21.961805 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:21.961769    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="80d35579-62f6-47a6-bcab-37f16bec576a" path="/var/lib/kubelet/pods/80d35579-62f6-47a6-bcab-37f16bec576a/volumes"
Oct 30 22:08:22.021609 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:22.021565    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4c260505-796c8fbff9-tdghh" podUID="80d35579-62f6-47a6-bcab-37f16bec576a" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.125:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:08:25.747316 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:25.747276    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:26.601746 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:26.601714    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:29.572555 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:29.572516    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:29.723130 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:29.723091    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:30.614002 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:30.613966    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:35.570275 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:08:35.570225    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:08:35.608290 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:35.608222    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:38.959984 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:38.959947    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:40.207497 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:40.207460    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:44.902405 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:44.902357    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:49.488743 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:49.488687    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:53.439595 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:53.439547    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:54.141861 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:54.141830    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:54.898444 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:54.898408    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:55.049129 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:55.049096    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:55.208065 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:55.207952    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:55.850658 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:08:55.850623    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:00.019933 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:00.019891    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:02.197699 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:02.197665    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:02.352775 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:02.352727    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:02.561157 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:02.561072    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"]
Oct 30 22:09:02.561328 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:02.561290    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm" podUID="f0cbb7ad-f150-4562-bb4c-ad9dce4545bd" containerName="istio-proxy" containerID="cri-o://2bfe9e580205f21e7c397d76d8dc80cb8774fbdb1a7f5ffc2b8690ae829b4fe4" gracePeriod=30
Oct 30 22:09:02.677274 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:02.656286    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:02.912868 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:02.912833    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:04.225336 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:04.225292    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:05.214328 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:05.214293    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:05.570542 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:09:05.570459    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:09:05.942707 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:05.942644    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:07.839167 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:07.839100    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"
Oct 30 22:09:07.939168 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:07.939130    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-istiod-ca-cert\") pod \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\" (UID: \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\") "
Oct 30 22:09:07.939372 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:07.939200    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-istio-envoy\") pod \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\" (UID: \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\") "
Oct 30 22:09:07.939372 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:07.939282    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-kube-api-access-7zqnl\" (UniqueName: \"kubernetes.io/projected/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-kube-api-access-7zqnl\") pod \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\" (UID: \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\") "
Oct 30 22:09:07.939372 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:07.939314    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-istio-data\") pod \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\" (UID: \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\") "
Oct 30 22:09:07.939372 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:07.939351    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-workload-socket\") pod \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\" (UID: \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\") "
Oct 30 22:09:07.939593 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:07.939391    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-istio-token\" (UniqueName: \"kubernetes.io/projected/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-istio-token\") pod \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\" (UID: \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\") "
Oct 30 22:09:07.939593 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:07.939468    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-workload-certs\") pod \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\" (UID: \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\") "
Oct 30 22:09:07.939703 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:07.939622    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-workload-socket" pod "f0cbb7ad-f150-4562-bb4c-ad9dce4545bd" (UID: "f0cbb7ad-f150-4562-bb4c-ad9dce4545bd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:07.939703 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:07.939639    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-credential-socket\") pod \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\" (UID: \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\") "
Oct 30 22:09:07.939703 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:07.939643    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-istio-data" pod "f0cbb7ad-f150-4562-bb4c-ad9dce4545bd" (UID: "f0cbb7ad-f150-4562-bb4c-ad9dce4545bd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:07.939703 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:07.939682    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-istio-podinfo\") pod \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\" (UID: \"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd\") "
Oct 30 22:09:07.939904 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:07.939832    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-workload-certs" pod "f0cbb7ad-f150-4562-bb4c-ad9dce4545bd" (UID: "f0cbb7ad-f150-4562-bb4c-ad9dce4545bd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:07.939973 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:07.939877    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-istiod-ca-cert" pod "f0cbb7ad-f150-4562-bb4c-ad9dce4545bd" (UID: "f0cbb7ad-f150-4562-bb4c-ad9dce4545bd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:09:07.939973 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:07.939943    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-credential-socket" pod "f0cbb7ad-f150-4562-bb4c-ad9dce4545bd" (UID: "f0cbb7ad-f150-4562-bb4c-ad9dce4545bd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:07.940383 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:07.940279    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:07.940383 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:07.940308    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:07.940383 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:07.940324    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:07.940383 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:07.940339    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:07.940383 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:07.940353    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:07.941708 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:07.941682    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-istio-envoy" pod "f0cbb7ad-f150-4562-bb4c-ad9dce4545bd" (UID: "f0cbb7ad-f150-4562-bb4c-ad9dce4545bd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:07.942011 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:07.941985    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-istio-token" pod "f0cbb7ad-f150-4562-bb4c-ad9dce4545bd" (UID: "f0cbb7ad-f150-4562-bb4c-ad9dce4545bd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:07.942078 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:07.942006    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-kube-api-access-7zqnl" pod "f0cbb7ad-f150-4562-bb4c-ad9dce4545bd" (UID: "f0cbb7ad-f150-4562-bb4c-ad9dce4545bd"). InnerVolumeSpecName "kube-api-access-7zqnl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:07.942078 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:07.942057    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-istio-podinfo" pod "f0cbb7ad-f150-4562-bb4c-ad9dce4545bd" (UID: "f0cbb7ad-f150-4562-bb4c-ad9dce4545bd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:09:08.041774 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:08.041735    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:08.041774 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:08.041774    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:08.042037 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:08.041789    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:08.042037 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:08.041804    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7zqnl\" (UniqueName: \"kubernetes.io/projected/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd-kube-api-access-7zqnl\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:08.521370 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:08.521335    4871 generic.go:356] "Generic (PLEG): container finished" podID="f0cbb7ad-f150-4562-bb4c-ad9dce4545bd" containerID="2bfe9e580205f21e7c397d76d8dc80cb8774fbdb1a7f5ffc2b8690ae829b4fe4" exitCode=0
Oct 30 22:09:08.521554 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:08.521390    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"
Oct 30 22:09:08.521554 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:08.521426    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm" event={"ID":"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd","Type":"ContainerDied","Data":"2bfe9e580205f21e7c397d76d8dc80cb8774fbdb1a7f5ffc2b8690ae829b4fe4"}
Oct 30 22:09:08.521554 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:08.521493    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm" event={"ID":"f0cbb7ad-f150-4562-bb4c-ad9dce4545bd","Type":"ContainerDied","Data":"0c920744aff9b96e6daaa5a41a472ef73531a4eb4127c7c9adcd2928dbc16ca2"}
Oct 30 22:09:08.521554 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:08.521519    4871 scope.go:122] "RemoveContainer" containerID="2bfe9e580205f21e7c397d76d8dc80cb8774fbdb1a7f5ffc2b8690ae829b4fe4"
Oct 30 22:09:08.532263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:08.532226    4871 scope.go:122] "RemoveContainer" containerID="d0b4495eb4008b7ed008ce1ded98730c0ab8dca8bdcb2dd342e6f7eed2fed0f3"
Oct 30 22:09:08.542123 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:08.542104    4871 scope.go:122] "RemoveContainer" containerID="2bfe9e580205f21e7c397d76d8dc80cb8774fbdb1a7f5ffc2b8690ae829b4fe4"
Oct 30 22:09:08.542455 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:09:08.542429    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bfe9e580205f21e7c397d76d8dc80cb8774fbdb1a7f5ffc2b8690ae829b4fe4\": container with ID starting with 2bfe9e580205f21e7c397d76d8dc80cb8774fbdb1a7f5ffc2b8690ae829b4fe4 not found: ID does not exist" containerID="2bfe9e580205f21e7c397d76d8dc80cb8774fbdb1a7f5ffc2b8690ae829b4fe4"
Oct 30 22:09:08.542560 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:08.542466    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bfe9e580205f21e7c397d76d8dc80cb8774fbdb1a7f5ffc2b8690ae829b4fe4"} err="failed to get container status \"2bfe9e580205f21e7c397d76d8dc80cb8774fbdb1a7f5ffc2b8690ae829b4fe4\": rpc error: code = NotFound desc = could not find container \"2bfe9e580205f21e7c397d76d8dc80cb8774fbdb1a7f5ffc2b8690ae829b4fe4\": container with ID starting with 2bfe9e580205f21e7c397d76d8dc80cb8774fbdb1a7f5ffc2b8690ae829b4fe4 not found: ID does not exist"
Oct 30 22:09:08.542560 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:08.542492    4871 scope.go:122] "RemoveContainer" containerID="d0b4495eb4008b7ed008ce1ded98730c0ab8dca8bdcb2dd342e6f7eed2fed0f3"
Oct 30 22:09:08.542799 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:09:08.542778    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0b4495eb4008b7ed008ce1ded98730c0ab8dca8bdcb2dd342e6f7eed2fed0f3\": container with ID starting with d0b4495eb4008b7ed008ce1ded98730c0ab8dca8bdcb2dd342e6f7eed2fed0f3 not found: ID does not exist" containerID="d0b4495eb4008b7ed008ce1ded98730c0ab8dca8bdcb2dd342e6f7eed2fed0f3"
Oct 30 22:09:08.542900 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:08.542804    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0b4495eb4008b7ed008ce1ded98730c0ab8dca8bdcb2dd342e6f7eed2fed0f3"} err="failed to get container status \"d0b4495eb4008b7ed008ce1ded98730c0ab8dca8bdcb2dd342e6f7eed2fed0f3\": rpc error: code = NotFound desc = could not find container \"d0b4495eb4008b7ed008ce1ded98730c0ab8dca8bdcb2dd342e6f7eed2fed0f3\": container with ID starting with d0b4495eb4008b7ed008ce1ded98730c0ab8dca8bdcb2dd342e6f7eed2fed0f3 not found: ID does not exist"
Oct 30 22:09:08.728720 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:08.728681    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"]
Oct 30 22:09:08.776331 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:08.776225    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-f9b7e7-openshift-default-558b7dc6c7-qw2bm"]
Oct 30 22:09:09.960695 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:09.960659    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f0cbb7ad-f150-4562-bb4c-ad9dce4545bd" path="/var/lib/kubelet/pods/f0cbb7ad-f150-4562-bb4c-ad9dce4545bd/volumes"
Oct 30 22:09:10.005668 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:10.005601    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:10.485324 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:10.485286    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:10.644301 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:10.644231    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:10.864416 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:10.864386    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"]
Oct 30 22:09:10.864658 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:10.864631    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw" podUID="957af2a6-bd01-49ed-a36a-1844daeebf2b" containerName="istio-proxy" containerID="cri-o://a2d4ab21d2a6b1d8cee1ea833a60a5d79c7adfc4cce028557f78d079de1228a1" gracePeriod=30
Oct 30 22:09:10.883320 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:10.883161    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"]
Oct 30 22:09:10.883562 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:10.883533    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz" podUID="198884dd-8c10-4d1f-9f64-6e00c0e7b6ab" containerName="istio-proxy" containerID="cri-o://baf0dc7131c0f6a4930be0190234056d6f1deaa5ff25fd7e2f2312b18e3406e4" gracePeriod=30
Oct 30 22:09:10.887271 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:10.887219    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"]
Oct 30 22:09:10.887442 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:10.887421    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns" podUID="a83867ab-e545-4023-8923-fc0124c9c638" containerName="istio-proxy" containerID="cri-o://749994a02830ffd008d548133f3d76b984fdfccfa1e442ba1cbb8c66a6d8a507" gracePeriod=30
Oct 30 22:09:10.894103 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:10.894078    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:11.757941 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:11.757910    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:12.312199 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.312159    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"]
Oct 30 22:09:12.312963 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.312941    4871 cpu_manager.go:414] "containerMap: removing container" podUID="ef03dee1-e693-42a0-ace8-4a28e099b92b" containerName="istio-proxy"
Oct 30 22:09:12.313026 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.312965    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ef03dee1-e693-42a0-ace8-4a28e099b92b" containerName="istio-proxy"
Oct 30 22:09:12.313026 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.312982    4871 cpu_manager.go:414] "containerMap: removing container" podUID="52b9b4dc-149f-4650-81f7-7f0234d9c6d8" containerName="istio-proxy"
Oct 30 22:09:12.313026 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.312988    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="52b9b4dc-149f-4650-81f7-7f0234d9c6d8" containerName="istio-proxy"
Oct 30 22:09:12.313026 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.313000    4871 cpu_manager.go:414] "containerMap: removing container" podUID="bc280bff-3593-4ae0-b27d-88339a1e100b" containerName="payload-processing"
Oct 30 22:09:12.313026 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.313005    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bc280bff-3593-4ae0-b27d-88339a1e100b" containerName="payload-processing"
Oct 30 22:09:12.313026 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.313012    4871 cpu_manager.go:414] "containerMap: removing container" podUID="9ea1b9a4-cdb1-475d-82f4-7b286e8410dc" containerName="maas-api"
Oct 30 22:09:12.313026 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.313017    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9ea1b9a4-cdb1-475d-82f4-7b286e8410dc" containerName="maas-api"
Oct 30 22:09:12.313026 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.313025    4871 cpu_manager.go:414] "containerMap: removing container" podUID="ffc32abd-61d0-447a-9050-2ad518ca585f" containerName="istio-proxy"
Oct 30 22:09:12.313026 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.313029    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ffc32abd-61d0-447a-9050-2ad518ca585f" containerName="istio-proxy"
Oct 30 22:09:12.313381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.313039    4871 cpu_manager.go:414] "containerMap: removing container" podUID="ef03dee1-e693-42a0-ace8-4a28e099b92b" containerName="istio-proxy"
Oct 30 22:09:12.313381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.313044    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ef03dee1-e693-42a0-ace8-4a28e099b92b" containerName="istio-proxy"
Oct 30 22:09:12.313381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.313052    4871 cpu_manager.go:414] "containerMap: removing container" podUID="f61805de-c5e9-42ef-9382-1e1c5c7d188e" containerName="maas-api"
Oct 30 22:09:12.313381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.313057    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f61805de-c5e9-42ef-9382-1e1c5c7d188e" containerName="maas-api"
Oct 30 22:09:12.313381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.313064    4871 cpu_manager.go:414] "containerMap: removing container" podUID="f0cbb7ad-f150-4562-bb4c-ad9dce4545bd" containerName="istio-proxy"
Oct 30 22:09:12.313381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.313071    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f0cbb7ad-f150-4562-bb4c-ad9dce4545bd" containerName="istio-proxy"
Oct 30 22:09:12.313381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.313084    4871 cpu_manager.go:414] "containerMap: removing container" podUID="19911666-f25a-4dd7-9789-7b9374b61cfb" containerName="istio-proxy"
Oct 30 22:09:12.313381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.313089    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="19911666-f25a-4dd7-9789-7b9374b61cfb" containerName="istio-proxy"
Oct 30 22:09:12.313381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.313096    4871 cpu_manager.go:414] "containerMap: removing container" podUID="48a6bcac-9719-4358-9019-71fd0a388ba1" containerName="istio-proxy"
Oct 30 22:09:12.313381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.313100    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="48a6bcac-9719-4358-9019-71fd0a388ba1" containerName="istio-proxy"
Oct 30 22:09:12.313381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.313111    4871 cpu_manager.go:414] "containerMap: removing container" podUID="b28e1c99-b48d-4a4e-a225-ebba779327e4" containerName="istio-proxy"
Oct 30 22:09:12.313381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.313115    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b28e1c99-b48d-4a4e-a225-ebba779327e4" containerName="istio-proxy"
Oct 30 22:09:12.313381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.313136    4871 cpu_manager.go:414] "containerMap: removing container" podUID="48a6bcac-9719-4358-9019-71fd0a388ba1" containerName="istio-proxy"
Oct 30 22:09:12.313381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.313141    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="48a6bcac-9719-4358-9019-71fd0a388ba1" containerName="istio-proxy"
Oct 30 22:09:12.313381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.313150    4871 cpu_manager.go:414] "containerMap: removing container" podUID="80d35579-62f6-47a6-bcab-37f16bec576a" containerName="maas-api"
Oct 30 22:09:12.313381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.313154    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="80d35579-62f6-47a6-bcab-37f16bec576a" containerName="maas-api"
Oct 30 22:09:12.313381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.313258    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="ef03dee1-e693-42a0-ace8-4a28e099b92b" containerName="istio-proxy"
Oct 30 22:09:12.313381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.313267    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="52b9b4dc-149f-4650-81f7-7f0234d9c6d8" containerName="istio-proxy"
Oct 30 22:09:12.313381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.313274    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="bc280bff-3593-4ae0-b27d-88339a1e100b" containerName="payload-processing"
Oct 30 22:09:12.313381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.313280    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="9ea1b9a4-cdb1-475d-82f4-7b286e8410dc" containerName="maas-api"
Oct 30 22:09:12.313381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.313289    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="ffc32abd-61d0-447a-9050-2ad518ca585f" containerName="istio-proxy"
Oct 30 22:09:12.313381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.313297    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="f61805de-c5e9-42ef-9382-1e1c5c7d188e" containerName="maas-api"
Oct 30 22:09:12.313381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.313303    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="f0cbb7ad-f150-4562-bb4c-ad9dce4545bd" containerName="istio-proxy"
Oct 30 22:09:12.313381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.313309    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="19911666-f25a-4dd7-9789-7b9374b61cfb" containerName="istio-proxy"
Oct 30 22:09:12.313381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.313314    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="48a6bcac-9719-4358-9019-71fd0a388ba1" containerName="istio-proxy"
Oct 30 22:09:12.313381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.313320    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="b28e1c99-b48d-4a4e-a225-ebba779327e4" containerName="istio-proxy"
Oct 30 22:09:12.313381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.313328    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="f0cbb7ad-f150-4562-bb4c-ad9dce4545bd" containerName="istio-proxy"
Oct 30 22:09:12.313381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.313334    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="80d35579-62f6-47a6-bcab-37f16bec576a" containerName="maas-api"
Oct 30 22:09:12.318839 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.318815    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"
Oct 30 22:09:12.499276 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.497821    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/088f4439-453d-406a-b54a-837e10b36efd-workload-certs\") pod \"e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh\" (UID: \"088f4439-453d-406a-b54a-837e10b36efd\") " pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"
Oct 30 22:09:12.499276 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.497938    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/088f4439-453d-406a-b54a-837e10b36efd-workload-socket\") pod \"e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh\" (UID: \"088f4439-453d-406a-b54a-837e10b36efd\") " pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"
Oct 30 22:09:12.499276 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.497987    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/088f4439-453d-406a-b54a-837e10b36efd-istiod-ca-cert\") pod \"e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh\" (UID: \"088f4439-453d-406a-b54a-837e10b36efd\") " pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"
Oct 30 22:09:12.499276 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.498023    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/088f4439-453d-406a-b54a-837e10b36efd-credential-socket\") pod \"e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh\" (UID: \"088f4439-453d-406a-b54a-837e10b36efd\") " pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"
Oct 30 22:09:12.499276 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.498073    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/088f4439-453d-406a-b54a-837e10b36efd-istio-data\") pod \"e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh\" (UID: \"088f4439-453d-406a-b54a-837e10b36efd\") " pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"
Oct 30 22:09:12.499276 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.498102    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/088f4439-453d-406a-b54a-837e10b36efd-istio-token\") pod \"e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh\" (UID: \"088f4439-453d-406a-b54a-837e10b36efd\") " pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"
Oct 30 22:09:12.499276 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.498128    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/088f4439-453d-406a-b54a-837e10b36efd-istio-podinfo\") pod \"e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh\" (UID: \"088f4439-453d-406a-b54a-837e10b36efd\") " pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"
Oct 30 22:09:12.499276 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.498198    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sx27c\" (UniqueName: \"kubernetes.io/projected/088f4439-453d-406a-b54a-837e10b36efd-kube-api-access-sx27c\") pod \"e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh\" (UID: \"088f4439-453d-406a-b54a-837e10b36efd\") " pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"
Oct 30 22:09:12.499276 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.498252    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/088f4439-453d-406a-b54a-837e10b36efd-istio-envoy\") pod \"e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh\" (UID: \"088f4439-453d-406a-b54a-837e10b36efd\") " pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"
Oct 30 22:09:12.524468 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.524426    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:12.525390 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.525305    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-52f191-openshift-default-dockercfg-jwp2h\""
Oct 30 22:09:12.589728 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.589631    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"]
Oct 30 22:09:12.624299 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.610955    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/088f4439-453d-406a-b54a-837e10b36efd-credential-socket\") pod \"e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh\" (UID: \"088f4439-453d-406a-b54a-837e10b36efd\") " pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"
Oct 30 22:09:12.624299 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.611189    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/088f4439-453d-406a-b54a-837e10b36efd-istio-data\") pod \"e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh\" (UID: \"088f4439-453d-406a-b54a-837e10b36efd\") " pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"
Oct 30 22:09:12.624299 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.620345    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/088f4439-453d-406a-b54a-837e10b36efd-credential-socket\") pod \"e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh\" (UID: \"088f4439-453d-406a-b54a-837e10b36efd\") " pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"
Oct 30 22:09:12.624299 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.621959    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/088f4439-453d-406a-b54a-837e10b36efd-istio-token\") pod \"e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh\" (UID: \"088f4439-453d-406a-b54a-837e10b36efd\") " pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"
Oct 30 22:09:12.624299 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.622227    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/088f4439-453d-406a-b54a-837e10b36efd-istio-podinfo\") pod \"e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh\" (UID: \"088f4439-453d-406a-b54a-837e10b36efd\") " pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"
Oct 30 22:09:12.624299 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.622531    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sx27c\" (UniqueName: \"kubernetes.io/projected/088f4439-453d-406a-b54a-837e10b36efd-kube-api-access-sx27c\") pod \"e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh\" (UID: \"088f4439-453d-406a-b54a-837e10b36efd\") " pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"
Oct 30 22:09:12.624299 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.622736    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/088f4439-453d-406a-b54a-837e10b36efd-istio-envoy\") pod \"e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh\" (UID: \"088f4439-453d-406a-b54a-837e10b36efd\") " pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"
Oct 30 22:09:12.624299 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.622974    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/088f4439-453d-406a-b54a-837e10b36efd-workload-certs\") pod \"e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh\" (UID: \"088f4439-453d-406a-b54a-837e10b36efd\") " pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"
Oct 30 22:09:12.624299 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.623146    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/088f4439-453d-406a-b54a-837e10b36efd-workload-socket\") pod \"e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh\" (UID: \"088f4439-453d-406a-b54a-837e10b36efd\") " pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"
Oct 30 22:09:12.624299 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.623411    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/088f4439-453d-406a-b54a-837e10b36efd-istiod-ca-cert\") pod \"e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh\" (UID: \"088f4439-453d-406a-b54a-837e10b36efd\") " pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"
Oct 30 22:09:12.634300 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.634219    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/088f4439-453d-406a-b54a-837e10b36efd-istio-podinfo\") pod \"e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh\" (UID: \"088f4439-453d-406a-b54a-837e10b36efd\") " pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"
Oct 30 22:09:12.638005 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.637972    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/088f4439-453d-406a-b54a-837e10b36efd-istio-data\") pod \"e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh\" (UID: \"088f4439-453d-406a-b54a-837e10b36efd\") " pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"
Oct 30 22:09:12.649006 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.648965    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/088f4439-453d-406a-b54a-837e10b36efd-workload-certs\") pod \"e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh\" (UID: \"088f4439-453d-406a-b54a-837e10b36efd\") " pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"
Oct 30 22:09:12.649508 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.649483    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/088f4439-453d-406a-b54a-837e10b36efd-workload-socket\") pod \"e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh\" (UID: \"088f4439-453d-406a-b54a-837e10b36efd\") " pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"
Oct 30 22:09:12.650070 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.649967    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/088f4439-453d-406a-b54a-837e10b36efd-istiod-ca-cert\") pod \"e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh\" (UID: \"088f4439-453d-406a-b54a-837e10b36efd\") " pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"
Oct 30 22:09:12.656907 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.656882    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/088f4439-453d-406a-b54a-837e10b36efd-istio-envoy\") pod \"e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh\" (UID: \"088f4439-453d-406a-b54a-837e10b36efd\") " pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"
Oct 30 22:09:12.694897 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.694551    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:12.699401 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.695190    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/088f4439-453d-406a-b54a-837e10b36efd-istio-token\") pod \"e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh\" (UID: \"088f4439-453d-406a-b54a-837e10b36efd\") " pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"
Oct 30 22:09:12.699401 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.697592    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sx27c\" (UniqueName: \"kubernetes.io/projected/088f4439-453d-406a-b54a-837e10b36efd-kube-api-access-sx27c\") pod \"e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh\" (UID: \"088f4439-453d-406a-b54a-837e10b36efd\") " pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"
Oct 30 22:09:12.891448 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:12.891409    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"
Oct 30 22:09:13.327278 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:13.326887    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"]
Oct 30 22:09:13.327278 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:09:13.327071    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod088f4439_453d_406a_b54a_837e10b36efd.slice/crio-b59d088c89d5fe943b57089bbaa55fad9b2ac2cda86b56f91670f5ca673b74fe WatchSource:0}: Error finding container b59d088c89d5fe943b57089bbaa55fad9b2ac2cda86b56f91670f5ca673b74fe: Status 404 returned error can't find the container with id b59d088c89d5fe943b57089bbaa55fad9b2ac2cda86b56f91670f5ca673b74fe
Oct 30 22:09:13.330124 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:13.330085    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:09:13.330257 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:13.330176    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:09:13.330257 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:13.330218    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:09:13.596705 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:13.596620    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh" event={"ID":"088f4439-453d-406a-b54a-837e10b36efd","Type":"ContainerStarted","Data":"df09c5f2757e1fed96666c817850397aac172e4d3efb7e932f8e6439ae47cc98"}
Oct 30 22:09:13.596705 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:13.596678    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh" event={"ID":"088f4439-453d-406a-b54a-837e10b36efd","Type":"ContainerStarted","Data":"b59d088c89d5fe943b57089bbaa55fad9b2ac2cda86b56f91670f5ca673b74fe"}
Oct 30 22:09:13.714920 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:13.714848    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh" podStartSLOduration=1.7148303299999998 podStartE2EDuration="1.71483033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:09:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:09:13.711310723 +0000 UTC m=+1895.888799344" watchObservedRunningTime="2026-10-30 22:09:13.71483033 +0000 UTC m=+1895.892318949"
Oct 30 22:09:13.892203 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:13.892159    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"
Oct 30 22:09:14.240083 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:14.239988    4871 patch_prober.go:29] interesting pod/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:09:14.240083 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:14.240048    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz" podUID="198884dd-8c10-4d1f-9f64-6e00c0e7b6ab" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:09:14.706457 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:14.706407    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:14.898064 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:14.898027    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"
Oct 30 22:09:15.286602 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:15.286565    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:15.379277 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:15.378170    4871 patch_prober.go:29] interesting pod/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:09:15.379277 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:15.378313    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw" podUID="957af2a6-bd01-49ed-a36a-1844daeebf2b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:09:15.396203 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:15.396168    4871 patch_prober.go:29] interesting pod/e2e-mt-4c260505-openshift-default-6f77466685-wcxns container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:09:15.400861 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:15.398118    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns" podUID="a83867ab-e545-4023-8923-fc0124c9c638" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:09:15.440724 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:15.440683    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:15.618692 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:15.618157    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"
Oct 30 22:09:15.619836 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:15.619801    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"
Oct 30 22:09:16.210450 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.210413    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"
Oct 30 22:09:16.261596 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.261566    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"
Oct 30 22:09:16.279850 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.279825    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"
Oct 30 22:09:16.306150 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.306108    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/957af2a6-bd01-49ed-a36a-1844daeebf2b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/957af2a6-bd01-49ed-a36a-1844daeebf2b-workload-certs\") pod \"957af2a6-bd01-49ed-a36a-1844daeebf2b\" (UID: \"957af2a6-bd01-49ed-a36a-1844daeebf2b\") "
Oct 30 22:09:16.306150 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.306145    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a83867ab-e545-4023-8923-fc0124c9c638-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a83867ab-e545-4023-8923-fc0124c9c638-istio-envoy\") pod \"a83867ab-e545-4023-8923-fc0124c9c638\" (UID: \"a83867ab-e545-4023-8923-fc0124c9c638\") "
Oct 30 22:09:16.306437 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.306203    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a83867ab-e545-4023-8923-fc0124c9c638-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a83867ab-e545-4023-8923-fc0124c9c638-istio-data\") pod \"a83867ab-e545-4023-8923-fc0124c9c638\" (UID: \"a83867ab-e545-4023-8923-fc0124c9c638\") "
Oct 30 22:09:16.306437 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.306424    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/957af2a6-bd01-49ed-a36a-1844daeebf2b-workload-certs" pod "957af2a6-bd01-49ed-a36a-1844daeebf2b" (UID: "957af2a6-bd01-49ed-a36a-1844daeebf2b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:16.306678 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.306229    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-workload-socket\") pod \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\" (UID: \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\") "
Oct 30 22:09:16.306741 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.306661    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-workload-socket" pod "198884dd-8c10-4d1f-9f64-6e00c0e7b6ab" (UID: "198884dd-8c10-4d1f-9f64-6e00c0e7b6ab"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:16.306797 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.306734    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/957af2a6-bd01-49ed-a36a-1844daeebf2b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/957af2a6-bd01-49ed-a36a-1844daeebf2b-istio-data\") pod \"957af2a6-bd01-49ed-a36a-1844daeebf2b\" (UID: \"957af2a6-bd01-49ed-a36a-1844daeebf2b\") "
Oct 30 22:09:16.306852 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.306801    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/957af2a6-bd01-49ed-a36a-1844daeebf2b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/957af2a6-bd01-49ed-a36a-1844daeebf2b-istio-podinfo\") pod \"957af2a6-bd01-49ed-a36a-1844daeebf2b\" (UID: \"957af2a6-bd01-49ed-a36a-1844daeebf2b\") "
Oct 30 22:09:16.306852 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.306716    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a83867ab-e545-4023-8923-fc0124c9c638-istio-data" pod "a83867ab-e545-4023-8923-fc0124c9c638" (UID: "a83867ab-e545-4023-8923-fc0124c9c638"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:16.306852 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.306836    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-istio-data\") pod \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\" (UID: \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\") "
Oct 30 22:09:16.307008 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.306874    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/957af2a6-bd01-49ed-a36a-1844daeebf2b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/957af2a6-bd01-49ed-a36a-1844daeebf2b-credential-socket\") pod \"957af2a6-bd01-49ed-a36a-1844daeebf2b\" (UID: \"957af2a6-bd01-49ed-a36a-1844daeebf2b\") "
Oct 30 22:09:16.307008 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.306906    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/957af2a6-bd01-49ed-a36a-1844daeebf2b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/957af2a6-bd01-49ed-a36a-1844daeebf2b-istiod-ca-cert\") pod \"957af2a6-bd01-49ed-a36a-1844daeebf2b\" (UID: \"957af2a6-bd01-49ed-a36a-1844daeebf2b\") "
Oct 30 22:09:16.307008 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.306982    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-istio-podinfo\") pod \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\" (UID: \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\") "
Oct 30 22:09:16.307163 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.307019    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a83867ab-e545-4023-8923-fc0124c9c638-kube-api-access-crfbm\" (UniqueName: \"kubernetes.io/projected/a83867ab-e545-4023-8923-fc0124c9c638-kube-api-access-crfbm\") pod \"a83867ab-e545-4023-8923-fc0124c9c638\" (UID: \"a83867ab-e545-4023-8923-fc0124c9c638\") "
Oct 30 22:09:16.307163 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.306906    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/957af2a6-bd01-49ed-a36a-1844daeebf2b-istio-data" pod "957af2a6-bd01-49ed-a36a-1844daeebf2b" (UID: "957af2a6-bd01-49ed-a36a-1844daeebf2b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:16.307163 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.307078    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-istio-data" pod "198884dd-8c10-4d1f-9f64-6e00c0e7b6ab" (UID: "198884dd-8c10-4d1f-9f64-6e00c0e7b6ab"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:16.307355 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.307318    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/957af2a6-bd01-49ed-a36a-1844daeebf2b-credential-socket" pod "957af2a6-bd01-49ed-a36a-1844daeebf2b" (UID: "957af2a6-bd01-49ed-a36a-1844daeebf2b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:16.307355 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.307084    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-kube-api-access-n9tbr\" (UniqueName: \"kubernetes.io/projected/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-kube-api-access-n9tbr\") pod \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\" (UID: \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\") "
Oct 30 22:09:16.307470 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.307412    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/957af2a6-bd01-49ed-a36a-1844daeebf2b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/957af2a6-bd01-49ed-a36a-1844daeebf2b-workload-socket\") pod \"957af2a6-bd01-49ed-a36a-1844daeebf2b\" (UID: \"957af2a6-bd01-49ed-a36a-1844daeebf2b\") "
Oct 30 22:09:16.307527 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.307465    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/957af2a6-bd01-49ed-a36a-1844daeebf2b-kube-api-access-sbc99\" (UniqueName: \"kubernetes.io/projected/957af2a6-bd01-49ed-a36a-1844daeebf2b-kube-api-access-sbc99\") pod \"957af2a6-bd01-49ed-a36a-1844daeebf2b\" (UID: \"957af2a6-bd01-49ed-a36a-1844daeebf2b\") "
Oct 30 22:09:16.307579 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.307526    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/957af2a6-bd01-49ed-a36a-1844daeebf2b-istiod-ca-cert" pod "957af2a6-bd01-49ed-a36a-1844daeebf2b" (UID: "957af2a6-bd01-49ed-a36a-1844daeebf2b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:09:16.307579 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.307544    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a83867ab-e545-4023-8923-fc0124c9c638-istio-token\" (UniqueName: \"kubernetes.io/projected/a83867ab-e545-4023-8923-fc0124c9c638-istio-token\") pod \"a83867ab-e545-4023-8923-fc0124c9c638\" (UID: \"a83867ab-e545-4023-8923-fc0124c9c638\") "
Oct 30 22:09:16.307689 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.307599    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-credential-socket\") pod \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\" (UID: \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\") "
Oct 30 22:09:16.307689 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.307636    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-istiod-ca-cert\") pod \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\" (UID: \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\") "
Oct 30 22:09:16.307689 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.307684    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/957af2a6-bd01-49ed-a36a-1844daeebf2b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/957af2a6-bd01-49ed-a36a-1844daeebf2b-istio-envoy\") pod \"957af2a6-bd01-49ed-a36a-1844daeebf2b\" (UID: \"957af2a6-bd01-49ed-a36a-1844daeebf2b\") "
Oct 30 22:09:16.307845 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.307723    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/957af2a6-bd01-49ed-a36a-1844daeebf2b-istio-token\" (UniqueName: \"kubernetes.io/projected/957af2a6-bd01-49ed-a36a-1844daeebf2b-istio-token\") pod \"957af2a6-bd01-49ed-a36a-1844daeebf2b\" (UID: \"957af2a6-bd01-49ed-a36a-1844daeebf2b\") "
Oct 30 22:09:16.307845 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.307781    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a83867ab-e545-4023-8923-fc0124c9c638-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a83867ab-e545-4023-8923-fc0124c9c638-workload-socket\") pod \"a83867ab-e545-4023-8923-fc0124c9c638\" (UID: \"a83867ab-e545-4023-8923-fc0124c9c638\") "
Oct 30 22:09:16.307845 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.307813    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a83867ab-e545-4023-8923-fc0124c9c638-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a83867ab-e545-4023-8923-fc0124c9c638-workload-certs\") pod \"a83867ab-e545-4023-8923-fc0124c9c638\" (UID: \"a83867ab-e545-4023-8923-fc0124c9c638\") "
Oct 30 22:09:16.308008 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.307843    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-istio-token\" (UniqueName: \"kubernetes.io/projected/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-istio-token\") pod \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\" (UID: \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\") "
Oct 30 22:09:16.308008 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.307889    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a83867ab-e545-4023-8923-fc0124c9c638-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a83867ab-e545-4023-8923-fc0124c9c638-istiod-ca-cert\") pod \"a83867ab-e545-4023-8923-fc0124c9c638\" (UID: \"a83867ab-e545-4023-8923-fc0124c9c638\") "
Oct 30 22:09:16.308008 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.307919    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-istio-envoy\") pod \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\" (UID: \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\") "
Oct 30 22:09:16.308008 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.307981    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a83867ab-e545-4023-8923-fc0124c9c638-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a83867ab-e545-4023-8923-fc0124c9c638-credential-socket\") pod \"a83867ab-e545-4023-8923-fc0124c9c638\" (UID: \"a83867ab-e545-4023-8923-fc0124c9c638\") "
Oct 30 22:09:16.308210 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.308012    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a83867ab-e545-4023-8923-fc0124c9c638-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a83867ab-e545-4023-8923-fc0124c9c638-istio-podinfo\") pod \"a83867ab-e545-4023-8923-fc0124c9c638\" (UID: \"a83867ab-e545-4023-8923-fc0124c9c638\") "
Oct 30 22:09:16.308210 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.308043    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-workload-certs\") pod \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\" (UID: \"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab\") "
Oct 30 22:09:16.308807 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.308783    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/957af2a6-bd01-49ed-a36a-1844daeebf2b-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:16.308998 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.308967    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a83867ab-e545-4023-8923-fc0124c9c638-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:16.308998 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.308995    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:16.309192 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.309009    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/957af2a6-bd01-49ed-a36a-1844daeebf2b-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:16.309192 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.309023    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:16.309192 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.309042    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/957af2a6-bd01-49ed-a36a-1844daeebf2b-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:16.309192 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.309057    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/957af2a6-bd01-49ed-a36a-1844daeebf2b-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:16.309461 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.309262    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-workload-certs" pod "198884dd-8c10-4d1f-9f64-6e00c0e7b6ab" (UID: "198884dd-8c10-4d1f-9f64-6e00c0e7b6ab"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:16.309461 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.309312    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/957af2a6-bd01-49ed-a36a-1844daeebf2b-workload-socket" pod "957af2a6-bd01-49ed-a36a-1844daeebf2b" (UID: "957af2a6-bd01-49ed-a36a-1844daeebf2b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:16.309461 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.309447    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-credential-socket" pod "198884dd-8c10-4d1f-9f64-6e00c0e7b6ab" (UID: "198884dd-8c10-4d1f-9f64-6e00c0e7b6ab"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:16.309637 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.309582    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a83867ab-e545-4023-8923-fc0124c9c638-workload-certs" pod "a83867ab-e545-4023-8923-fc0124c9c638" (UID: "a83867ab-e545-4023-8923-fc0124c9c638"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:16.310023 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.309922    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-istiod-ca-cert" pod "198884dd-8c10-4d1f-9f64-6e00c0e7b6ab" (UID: "198884dd-8c10-4d1f-9f64-6e00c0e7b6ab"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:09:16.310219 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.310193    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a83867ab-e545-4023-8923-fc0124c9c638-credential-socket" pod "a83867ab-e545-4023-8923-fc0124c9c638" (UID: "a83867ab-e545-4023-8923-fc0124c9c638"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:16.310644 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.310615    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a83867ab-e545-4023-8923-fc0124c9c638-istio-envoy" pod "a83867ab-e545-4023-8923-fc0124c9c638" (UID: "a83867ab-e545-4023-8923-fc0124c9c638"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:16.311190 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.310993    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a83867ab-e545-4023-8923-fc0124c9c638-istiod-ca-cert" pod "a83867ab-e545-4023-8923-fc0124c9c638" (UID: "a83867ab-e545-4023-8923-fc0124c9c638"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:09:16.311413 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.311367    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a83867ab-e545-4023-8923-fc0124c9c638-workload-socket" pod "a83867ab-e545-4023-8923-fc0124c9c638" (UID: "a83867ab-e545-4023-8923-fc0124c9c638"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:16.313199 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.313042    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/957af2a6-bd01-49ed-a36a-1844daeebf2b-istio-podinfo" pod "957af2a6-bd01-49ed-a36a-1844daeebf2b" (UID: "957af2a6-bd01-49ed-a36a-1844daeebf2b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:09:16.313199 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.313106    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-istio-podinfo" pod "198884dd-8c10-4d1f-9f64-6e00c0e7b6ab" (UID: "198884dd-8c10-4d1f-9f64-6e00c0e7b6ab"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:09:16.313519 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.313469    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-istio-envoy" pod "198884dd-8c10-4d1f-9f64-6e00c0e7b6ab" (UID: "198884dd-8c10-4d1f-9f64-6e00c0e7b6ab"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:16.313782 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.313749    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a83867ab-e545-4023-8923-fc0124c9c638-kube-api-access-crfbm" pod "a83867ab-e545-4023-8923-fc0124c9c638" (UID: "a83867ab-e545-4023-8923-fc0124c9c638"). InnerVolumeSpecName "kube-api-access-crfbm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:16.313876 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.313827    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-istio-token" pod "198884dd-8c10-4d1f-9f64-6e00c0e7b6ab" (UID: "198884dd-8c10-4d1f-9f64-6e00c0e7b6ab"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:16.314178 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.314045    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-kube-api-access-n9tbr" pod "198884dd-8c10-4d1f-9f64-6e00c0e7b6ab" (UID: "198884dd-8c10-4d1f-9f64-6e00c0e7b6ab"). InnerVolumeSpecName "kube-api-access-n9tbr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:16.314178 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.314135    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a83867ab-e545-4023-8923-fc0124c9c638-istio-podinfo" pod "a83867ab-e545-4023-8923-fc0124c9c638" (UID: "a83867ab-e545-4023-8923-fc0124c9c638"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:09:16.314473 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.314441    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/957af2a6-bd01-49ed-a36a-1844daeebf2b-istio-token" pod "957af2a6-bd01-49ed-a36a-1844daeebf2b" (UID: "957af2a6-bd01-49ed-a36a-1844daeebf2b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:16.314558 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.314538    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/957af2a6-bd01-49ed-a36a-1844daeebf2b-kube-api-access-sbc99" pod "957af2a6-bd01-49ed-a36a-1844daeebf2b" (UID: "957af2a6-bd01-49ed-a36a-1844daeebf2b"). InnerVolumeSpecName "kube-api-access-sbc99". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:16.314916 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.314893    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/957af2a6-bd01-49ed-a36a-1844daeebf2b-istio-envoy" pod "957af2a6-bd01-49ed-a36a-1844daeebf2b" (UID: "957af2a6-bd01-49ed-a36a-1844daeebf2b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:16.315398 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.315379    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a83867ab-e545-4023-8923-fc0124c9c638-istio-token" pod "a83867ab-e545-4023-8923-fc0124c9c638" (UID: "a83867ab-e545-4023-8923-fc0124c9c638"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:16.422312 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.417398    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/957af2a6-bd01-49ed-a36a-1844daeebf2b-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:16.422312 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.417441    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a83867ab-e545-4023-8923-fc0124c9c638-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:16.422312 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.417458    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a83867ab-e545-4023-8923-fc0124c9c638-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:16.422312 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.417480    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:16.422312 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.417494    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a83867ab-e545-4023-8923-fc0124c9c638-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:16.422312 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.417510    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:16.422312 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.417524    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a83867ab-e545-4023-8923-fc0124c9c638-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:16.422312 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.417537    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a83867ab-e545-4023-8923-fc0124c9c638-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:16.422312 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.417551    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:16.422312 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.417566    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a83867ab-e545-4023-8923-fc0124c9c638-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:16.422312 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.417579    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/957af2a6-bd01-49ed-a36a-1844daeebf2b-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:16.422312 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.417593    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:16.422312 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.417606    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crfbm\" (UniqueName: \"kubernetes.io/projected/a83867ab-e545-4023-8923-fc0124c9c638-kube-api-access-crfbm\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:16.422312 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.417620    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9tbr\" (UniqueName: \"kubernetes.io/projected/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-kube-api-access-n9tbr\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:16.422312 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.417633    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/957af2a6-bd01-49ed-a36a-1844daeebf2b-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:16.422312 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.417646    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbc99\" (UniqueName: \"kubernetes.io/projected/957af2a6-bd01-49ed-a36a-1844daeebf2b-kube-api-access-sbc99\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:16.422312 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.417661    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a83867ab-e545-4023-8923-fc0124c9c638-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:16.422312 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.417678    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:16.422312 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.417692    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:16.422312 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.417705    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/957af2a6-bd01-49ed-a36a-1844daeebf2b-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:16.595605 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.571365    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:16.634195 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.634150    4871 generic.go:356] "Generic (PLEG): container finished" podID="a83867ab-e545-4023-8923-fc0124c9c638" containerID="749994a02830ffd008d548133f3d76b984fdfccfa1e442ba1cbb8c66a6d8a507" exitCode=0
Oct 30 22:09:16.634535 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.634219    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"
Oct 30 22:09:16.634628 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.634258    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns" event={"ID":"a83867ab-e545-4023-8923-fc0124c9c638","Type":"ContainerDied","Data":"749994a02830ffd008d548133f3d76b984fdfccfa1e442ba1cbb8c66a6d8a507"}
Oct 30 22:09:16.634692 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.634657    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns" event={"ID":"a83867ab-e545-4023-8923-fc0124c9c638","Type":"ContainerDied","Data":"0e7b2802045f5ff77fe9e79451a3274ac0749ad97715c446c7dc7d308ce66a5d"}
Oct 30 22:09:16.634750 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.634693    4871 scope.go:122] "RemoveContainer" containerID="749994a02830ffd008d548133f3d76b984fdfccfa1e442ba1cbb8c66a6d8a507"
Oct 30 22:09:16.643270 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.639465    4871 generic.go:356] "Generic (PLEG): container finished" podID="198884dd-8c10-4d1f-9f64-6e00c0e7b6ab" containerID="baf0dc7131c0f6a4930be0190234056d6f1deaa5ff25fd7e2f2312b18e3406e4" exitCode=0
Oct 30 22:09:16.643270 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.639578    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz" event={"ID":"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab","Type":"ContainerDied","Data":"baf0dc7131c0f6a4930be0190234056d6f1deaa5ff25fd7e2f2312b18e3406e4"}
Oct 30 22:09:16.643270 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.639625    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz" event={"ID":"198884dd-8c10-4d1f-9f64-6e00c0e7b6ab","Type":"ContainerDied","Data":"ee53d1b1dc83a8ec70ec8053b84c8575a79789082c181c9829659b6dab06bb8d"}
Oct 30 22:09:16.643270 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.639715    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"
Oct 30 22:09:16.664505 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.660031    4871 generic.go:356] "Generic (PLEG): container finished" podID="957af2a6-bd01-49ed-a36a-1844daeebf2b" containerID="a2d4ab21d2a6b1d8cee1ea833a60a5d79c7adfc4cce028557f78d079de1228a1" exitCode=0
Oct 30 22:09:16.664505 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.661168    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"
Oct 30 22:09:16.665656 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.665205    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw" event={"ID":"957af2a6-bd01-49ed-a36a-1844daeebf2b","Type":"ContainerDied","Data":"a2d4ab21d2a6b1d8cee1ea833a60a5d79c7adfc4cce028557f78d079de1228a1"}
Oct 30 22:09:16.665656 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.665288    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw" event={"ID":"957af2a6-bd01-49ed-a36a-1844daeebf2b","Type":"ContainerDied","Data":"bb4005ad98edaffbcf60552898a918d18c4dc8db58f04af501f9ad0cbd978c97"}
Oct 30 22:09:16.666996 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.666972    4871 scope.go:122] "RemoveContainer" containerID="749994a02830ffd008d548133f3d76b984fdfccfa1e442ba1cbb8c66a6d8a507"
Oct 30 22:09:16.667392 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:09:16.667367    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"749994a02830ffd008d548133f3d76b984fdfccfa1e442ba1cbb8c66a6d8a507\": container with ID starting with 749994a02830ffd008d548133f3d76b984fdfccfa1e442ba1cbb8c66a6d8a507 not found: ID does not exist" containerID="749994a02830ffd008d548133f3d76b984fdfccfa1e442ba1cbb8c66a6d8a507"
Oct 30 22:09:16.667487 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.667403    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"749994a02830ffd008d548133f3d76b984fdfccfa1e442ba1cbb8c66a6d8a507"} err="failed to get container status \"749994a02830ffd008d548133f3d76b984fdfccfa1e442ba1cbb8c66a6d8a507\": rpc error: code = NotFound desc = could not find container \"749994a02830ffd008d548133f3d76b984fdfccfa1e442ba1cbb8c66a6d8a507\": container with ID starting with 749994a02830ffd008d548133f3d76b984fdfccfa1e442ba1cbb8c66a6d8a507 not found: ID does not exist"
Oct 30 22:09:16.667487 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.667425    4871 scope.go:122] "RemoveContainer" containerID="baf0dc7131c0f6a4930be0190234056d6f1deaa5ff25fd7e2f2312b18e3406e4"
Oct 30 22:09:16.681842 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.681823    4871 scope.go:122] "RemoveContainer" containerID="b419997ba915969d7decd8f7a38ecdfe34dca49729e2b8747e3e95f3ea373138"
Oct 30 22:09:16.695734 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.695713    4871 scope.go:122] "RemoveContainer" containerID="baf0dc7131c0f6a4930be0190234056d6f1deaa5ff25fd7e2f2312b18e3406e4"
Oct 30 22:09:16.696102 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:09:16.696076    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"baf0dc7131c0f6a4930be0190234056d6f1deaa5ff25fd7e2f2312b18e3406e4\": container with ID starting with baf0dc7131c0f6a4930be0190234056d6f1deaa5ff25fd7e2f2312b18e3406e4 not found: ID does not exist" containerID="baf0dc7131c0f6a4930be0190234056d6f1deaa5ff25fd7e2f2312b18e3406e4"
Oct 30 22:09:16.696220 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.696111    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"baf0dc7131c0f6a4930be0190234056d6f1deaa5ff25fd7e2f2312b18e3406e4"} err="failed to get container status \"baf0dc7131c0f6a4930be0190234056d6f1deaa5ff25fd7e2f2312b18e3406e4\": rpc error: code = NotFound desc = could not find container \"baf0dc7131c0f6a4930be0190234056d6f1deaa5ff25fd7e2f2312b18e3406e4\": container with ID starting with baf0dc7131c0f6a4930be0190234056d6f1deaa5ff25fd7e2f2312b18e3406e4 not found: ID does not exist"
Oct 30 22:09:16.696220 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.696135    4871 scope.go:122] "RemoveContainer" containerID="b419997ba915969d7decd8f7a38ecdfe34dca49729e2b8747e3e95f3ea373138"
Oct 30 22:09:16.696511 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:09:16.696491    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b419997ba915969d7decd8f7a38ecdfe34dca49729e2b8747e3e95f3ea373138\": container with ID starting with b419997ba915969d7decd8f7a38ecdfe34dca49729e2b8747e3e95f3ea373138 not found: ID does not exist" containerID="b419997ba915969d7decd8f7a38ecdfe34dca49729e2b8747e3e95f3ea373138"
Oct 30 22:09:16.696574 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.696516    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b419997ba915969d7decd8f7a38ecdfe34dca49729e2b8747e3e95f3ea373138"} err="failed to get container status \"b419997ba915969d7decd8f7a38ecdfe34dca49729e2b8747e3e95f3ea373138\": rpc error: code = NotFound desc = could not find container \"b419997ba915969d7decd8f7a38ecdfe34dca49729e2b8747e3e95f3ea373138\": container with ID starting with b419997ba915969d7decd8f7a38ecdfe34dca49729e2b8747e3e95f3ea373138 not found: ID does not exist"
Oct 30 22:09:16.696574 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.696534    4871 scope.go:122] "RemoveContainer" containerID="a2d4ab21d2a6b1d8cee1ea833a60a5d79c7adfc4cce028557f78d079de1228a1"
Oct 30 22:09:16.709676 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.709651    4871 scope.go:122] "RemoveContainer" containerID="a2d4ab21d2a6b1d8cee1ea833a60a5d79c7adfc4cce028557f78d079de1228a1"
Oct 30 22:09:16.710115 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:09:16.710088    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2d4ab21d2a6b1d8cee1ea833a60a5d79c7adfc4cce028557f78d079de1228a1\": container with ID starting with a2d4ab21d2a6b1d8cee1ea833a60a5d79c7adfc4cce028557f78d079de1228a1 not found: ID does not exist" containerID="a2d4ab21d2a6b1d8cee1ea833a60a5d79c7adfc4cce028557f78d079de1228a1"
Oct 30 22:09:16.710199 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.710132    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2d4ab21d2a6b1d8cee1ea833a60a5d79c7adfc4cce028557f78d079de1228a1"} err="failed to get container status \"a2d4ab21d2a6b1d8cee1ea833a60a5d79c7adfc4cce028557f78d079de1228a1\": rpc error: code = NotFound desc = could not find container \"a2d4ab21d2a6b1d8cee1ea833a60a5d79c7adfc4cce028557f78d079de1228a1\": container with ID starting with a2d4ab21d2a6b1d8cee1ea833a60a5d79c7adfc4cce028557f78d079de1228a1 not found: ID does not exist"
Oct 30 22:09:16.737782 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.737743    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:16.881814 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.881746    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"]
Oct 30 22:09:16.946652 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.946376    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-wcxns"]
Oct 30 22:09:16.970661 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.970601    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"]
Oct 30 22:09:16.975270 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:16.975195    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-j6sfz"]
Oct 30 22:09:17.048870 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:17.048833    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"]
Oct 30 22:09:17.056603 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:17.056567    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-4c260505-openshift-default-6f77466685-xdbqw"]
Oct 30 22:09:17.960755 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:17.960724    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="198884dd-8c10-4d1f-9f64-6e00c0e7b6ab" path="/var/lib/kubelet/pods/198884dd-8c10-4d1f-9f64-6e00c0e7b6ab/volumes"
Oct 30 22:09:17.961327 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:17.961310    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="957af2a6-bd01-49ed-a36a-1844daeebf2b" path="/var/lib/kubelet/pods/957af2a6-bd01-49ed-a36a-1844daeebf2b/volumes"
Oct 30 22:09:17.961803 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:17.961789    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a83867ab-e545-4023-8923-fc0124c9c638" path="/var/lib/kubelet/pods/a83867ab-e545-4023-8923-fc0124c9c638/volumes"
Oct 30 22:09:18.844663 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:18.844616    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5"]
Oct 30 22:09:18.844871 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:18.844846    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5" podUID="904ace00-01d5-4e6b-ba97-d9e887b5da40" containerName="maas-api" containerID="cri-o://59ed94f8f793d33a440af240a0a4add3d10510e0726aebf056b32e7dabb00ea7" gracePeriod=30
Oct 30 22:09:18.970574 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:18.970140    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:19.199309 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:19.199218    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5"
Oct 30 22:09:19.301064 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:19.301030    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/904ace00-01d5-4e6b-ba97-d9e887b5da40-kube-api-access-d5xgk\" (UniqueName: \"kubernetes.io/projected/904ace00-01d5-4e6b-ba97-d9e887b5da40-kube-api-access-d5xgk\") pod \"904ace00-01d5-4e6b-ba97-d9e887b5da40\" (UID: \"904ace00-01d5-4e6b-ba97-d9e887b5da40\") "
Oct 30 22:09:19.301291 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:19.301225    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/904ace00-01d5-4e6b-ba97-d9e887b5da40-metrics-certs\" (UniqueName: \"kubernetes.io/secret/904ace00-01d5-4e6b-ba97-d9e887b5da40-metrics-certs\") pod \"904ace00-01d5-4e6b-ba97-d9e887b5da40\" (UID: \"904ace00-01d5-4e6b-ba97-d9e887b5da40\") "
Oct 30 22:09:19.301373 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:19.301327    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/904ace00-01d5-4e6b-ba97-d9e887b5da40-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/904ace00-01d5-4e6b-ba97-d9e887b5da40-maas-api-tls\") pod \"904ace00-01d5-4e6b-ba97-d9e887b5da40\" (UID: \"904ace00-01d5-4e6b-ba97-d9e887b5da40\") "
Oct 30 22:09:19.301429 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:19.301417    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/904ace00-01d5-4e6b-ba97-d9e887b5da40-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/904ace00-01d5-4e6b-ba97-d9e887b5da40-openshift-service-ca\") pod \"904ace00-01d5-4e6b-ba97-d9e887b5da40\" (UID: \"904ace00-01d5-4e6b-ba97-d9e887b5da40\") "
Oct 30 22:09:19.302017 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:19.301984    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/904ace00-01d5-4e6b-ba97-d9e887b5da40-openshift-service-ca" pod "904ace00-01d5-4e6b-ba97-d9e887b5da40" (UID: "904ace00-01d5-4e6b-ba97-d9e887b5da40"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:09:19.303529 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:19.303502    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/904ace00-01d5-4e6b-ba97-d9e887b5da40-maas-api-tls" pod "904ace00-01d5-4e6b-ba97-d9e887b5da40" (UID: "904ace00-01d5-4e6b-ba97-d9e887b5da40"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:09:19.303617 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:19.303548    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/904ace00-01d5-4e6b-ba97-d9e887b5da40-kube-api-access-d5xgk" pod "904ace00-01d5-4e6b-ba97-d9e887b5da40" (UID: "904ace00-01d5-4e6b-ba97-d9e887b5da40"). InnerVolumeSpecName "kube-api-access-d5xgk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:19.303617 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:19.303570    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/904ace00-01d5-4e6b-ba97-d9e887b5da40-metrics-certs" pod "904ace00-01d5-4e6b-ba97-d9e887b5da40" (UID: "904ace00-01d5-4e6b-ba97-d9e887b5da40"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:09:19.402987 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:19.402952    4871 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/904ace00-01d5-4e6b-ba97-d9e887b5da40-metrics-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:19.402987 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:19.402983    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/904ace00-01d5-4e6b-ba97-d9e887b5da40-maas-api-tls\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:19.402987 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:19.402995    4871 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/904ace00-01d5-4e6b-ba97-d9e887b5da40-openshift-service-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:19.403225 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:19.403013    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5xgk\" (UniqueName: \"kubernetes.io/projected/904ace00-01d5-4e6b-ba97-d9e887b5da40-kube-api-access-d5xgk\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:19.688163 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:19.688127    4871 generic.go:356] "Generic (PLEG): container finished" podID="904ace00-01d5-4e6b-ba97-d9e887b5da40" containerID="59ed94f8f793d33a440af240a0a4add3d10510e0726aebf056b32e7dabb00ea7" exitCode=0
Oct 30 22:09:19.688439 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:19.688186    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5"
Oct 30 22:09:19.688439 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:19.688201    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5" event={"ID":"904ace00-01d5-4e6b-ba97-d9e887b5da40","Type":"ContainerDied","Data":"59ed94f8f793d33a440af240a0a4add3d10510e0726aebf056b32e7dabb00ea7"}
Oct 30 22:09:19.688439 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:19.688291    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5" event={"ID":"904ace00-01d5-4e6b-ba97-d9e887b5da40","Type":"ContainerDied","Data":"c174006530408c291104ccb376da07b6c60bbc0c7eba254e37f33a90184dd128"}
Oct 30 22:09:19.688439 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:19.688318    4871 scope.go:122] "RemoveContainer" containerID="59ed94f8f793d33a440af240a0a4add3d10510e0726aebf056b32e7dabb00ea7"
Oct 30 22:09:19.700802 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:19.700777    4871 scope.go:122] "RemoveContainer" containerID="59ed94f8f793d33a440af240a0a4add3d10510e0726aebf056b32e7dabb00ea7"
Oct 30 22:09:19.701116 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:09:19.701089    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59ed94f8f793d33a440af240a0a4add3d10510e0726aebf056b32e7dabb00ea7\": container with ID starting with 59ed94f8f793d33a440af240a0a4add3d10510e0726aebf056b32e7dabb00ea7 not found: ID does not exist" containerID="59ed94f8f793d33a440af240a0a4add3d10510e0726aebf056b32e7dabb00ea7"
Oct 30 22:09:19.701197 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:19.701126    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59ed94f8f793d33a440af240a0a4add3d10510e0726aebf056b32e7dabb00ea7"} err="failed to get container status \"59ed94f8f793d33a440af240a0a4add3d10510e0726aebf056b32e7dabb00ea7\": rpc error: code = NotFound desc = could not find container \"59ed94f8f793d33a440af240a0a4add3d10510e0726aebf056b32e7dabb00ea7\": container with ID starting with 59ed94f8f793d33a440af240a0a4add3d10510e0726aebf056b32e7dabb00ea7 not found: ID does not exist"
Oct 30 22:09:19.731080 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:19.731042    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:19.887260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:19.885285    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5"]
Oct 30 22:09:19.964313 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:19.930873    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a5f7b9-776b8457fd-t5qw5"]
Oct 30 22:09:19.964313 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:19.931937    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:20.141711 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:20.141664    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="904ace00-01d5-4e6b-ba97-d9e887b5da40" path="/var/lib/kubelet/pods/904ace00-01d5-4e6b-ba97-d9e887b5da40/volumes"
Oct 30 22:09:20.207764 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:20.207729    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5"]
Oct 30 22:09:20.208077 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:20.208036    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5" podUID="0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd" containerName="maas-api" containerID="cri-o://63357b25bf278c08446c7c3f842590dd72af36fd3349a44fedf3ed59f8c0f0e5" gracePeriod=30
Oct 30 22:09:20.316051 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:20.316013    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:20.782896 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:20.782541    4871 generic.go:356] "Generic (PLEG): container finished" podID="0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd" containerID="63357b25bf278c08446c7c3f842590dd72af36fd3349a44fedf3ed59f8c0f0e5" exitCode=0
Oct 30 22:09:20.782896 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:20.782796    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:20.782896 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:20.782833    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5" event={"ID":"0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd","Type":"ContainerDied","Data":"63357b25bf278c08446c7c3f842590dd72af36fd3349a44fedf3ed59f8c0f0e5"}
Oct 30 22:09:20.823268 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:20.821278    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5"
Oct 30 22:09:20.863864 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:20.863830    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd-metrics-certs\") pod \"0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd\" (UID: \"0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd\") "
Oct 30 22:09:20.864022 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:20.863967    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd-openshift-service-ca\") pod \"0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd\" (UID: \"0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd\") "
Oct 30 22:09:20.864022 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:20.864014    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd-kube-api-access-mljph\" (UniqueName: \"kubernetes.io/projected/0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd-kube-api-access-mljph\") pod \"0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd\" (UID: \"0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd\") "
Oct 30 22:09:20.864126 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:20.864061    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd-maas-api-tls\") pod \"0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd\" (UID: \"0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd\") "
Oct 30 22:09:20.864445 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:20.864403    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd-openshift-service-ca" pod "0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd" (UID: "0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:09:20.864694 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:20.864674    4871 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd-openshift-service-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:20.866152 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:20.866121    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd-metrics-certs" pod "0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd" (UID: "0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:09:20.866305 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:20.866182    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd-maas-api-tls" pod "0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd" (UID: "0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:09:20.866392 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:20.866375    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd-kube-api-access-mljph" pod "0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd" (UID: "0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd"). InnerVolumeSpecName "kube-api-access-mljph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:20.966290 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:20.966219    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mljph\" (UniqueName: \"kubernetes.io/projected/0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd-kube-api-access-mljph\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:20.966290 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:20.966286    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd-maas-api-tls\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:20.966290 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:20.966298    4871 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd-metrics-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:21.042264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:21.042169    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:21.697037 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:21.696982    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:21.793606 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:21.793577    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5"
Oct 30 22:09:21.793606 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:21.793591    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5" event={"ID":"0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd","Type":"ContainerDied","Data":"2909a78aac91cf84b036ac584830e0b5c913501fa53c78fe461c448a424406ba"}
Oct 30 22:09:21.793804 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:21.793657    4871 scope.go:122] "RemoveContainer" containerID="63357b25bf278c08446c7c3f842590dd72af36fd3349a44fedf3ed59f8c0f0e5"
Oct 30 22:09:21.968600 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:21.968499    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:21.977620 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:21.977590    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5"]
Oct 30 22:09:22.001280 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:22.001229    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4cb675-f9f78c55b-qp9h5"]
Oct 30 22:09:22.220720 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:22.220595    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:22.372848 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:22.372794    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:22.938030 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:22.938001    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:23.964183 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:23.964139    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd" path="/var/lib/kubelet/pods/0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd/volumes"
Oct 30 22:09:27.201973 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:27.201934    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:27.888408 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:27.888357    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:28.054942 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:28.054905    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:28.250724 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:28.250641    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"]
Oct 30 22:09:28.251127 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:28.250896    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp" podUID="e00cf9ef-6182-4fc8-b564-ea2425c9e5a5" containerName="istio-proxy" containerID="cri-o://02d412827d862380b10acbe699f6fcce68156210b81c32b1fa8c2a28c81e62db" gracePeriod=30
Oct 30 22:09:28.292788 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:28.292743    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:29.728358 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:29.728318    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:29.918732 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:29.918696    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:30.112128 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:30.112086    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"]
Oct 30 22:09:30.115185 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:30.114084    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l" podUID="d5e633ba-5844-4f32-956d-8757ce76acce" containerName="istio-proxy" containerID="cri-o://f30ddb4ace4b9375945ae44f718c1761e69dfb42111929ccac7ab160d117e158" gracePeriod=30
Oct 30 22:09:30.153034 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:30.152998    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:31.261570 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:31.261526    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:32.378922 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:32.378882    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"]
Oct 30 22:09:32.379415 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:32.379094    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k" podUID="c5b0cd90-fe64-4446-9a53-9ab9ebaa269e" containerName="istio-proxy" containerID="cri-o://5684f813bd9bb13ddba214ca358f59f5f40f290556249dd8396053a96cba135f" gracePeriod=30
Oct 30 22:09:32.779738 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:32.779661    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:32.994411 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:32.994376    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:33.152684 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:33.152654    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:33.547651 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:33.547622    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"
Oct 30 22:09:33.586017 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:33.585981    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-istio-podinfo\") pod \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\" (UID: \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\") "
Oct 30 22:09:33.586225 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:33.586038    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-istiod-ca-cert\") pod \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\" (UID: \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\") "
Oct 30 22:09:33.586225 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:33.586069    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-istio-token\" (UniqueName: \"kubernetes.io/projected/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-istio-token\") pod \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\" (UID: \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\") "
Oct 30 22:09:33.586225 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:33.586172    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-kube-api-access-d872l\" (UniqueName: \"kubernetes.io/projected/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-kube-api-access-d872l\") pod \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\" (UID: \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\") "
Oct 30 22:09:33.586429 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:33.586270    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-istio-data\") pod \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\" (UID: \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\") "
Oct 30 22:09:33.586429 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:33.586369    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-workload-socket\") pod \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\" (UID: \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\") "
Oct 30 22:09:33.586429 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:33.586424    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-credential-socket\") pod \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\" (UID: \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\") "
Oct 30 22:09:33.586592 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:33.586459    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-istio-envoy\") pod \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\" (UID: \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\") "
Oct 30 22:09:33.586592 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:33.586550    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-workload-certs\") pod \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\" (UID: \"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5\") "
Oct 30 22:09:33.586713 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:33.586589    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-istiod-ca-cert" pod "e00cf9ef-6182-4fc8-b564-ea2425c9e5a5" (UID: "e00cf9ef-6182-4fc8-b564-ea2425c9e5a5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:09:33.587269 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:33.586748    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-workload-socket" pod "e00cf9ef-6182-4fc8-b564-ea2425c9e5a5" (UID: "e00cf9ef-6182-4fc8-b564-ea2425c9e5a5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:33.587269 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:33.586823    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-credential-socket" pod "e00cf9ef-6182-4fc8-b564-ea2425c9e5a5" (UID: "e00cf9ef-6182-4fc8-b564-ea2425c9e5a5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:33.587269 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:33.586883    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-istio-data" pod "e00cf9ef-6182-4fc8-b564-ea2425c9e5a5" (UID: "e00cf9ef-6182-4fc8-b564-ea2425c9e5a5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:33.587269 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:33.587053    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-workload-certs" pod "e00cf9ef-6182-4fc8-b564-ea2425c9e5a5" (UID: "e00cf9ef-6182-4fc8-b564-ea2425c9e5a5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:33.587269 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:33.587177    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:33.587269 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:33.587193    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:33.587269 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:33.587202    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:33.587269 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:33.587216    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:33.587269 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:33.587229    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:33.588977 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:33.588948    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-istio-envoy" pod "e00cf9ef-6182-4fc8-b564-ea2425c9e5a5" (UID: "e00cf9ef-6182-4fc8-b564-ea2425c9e5a5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:33.588977 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:33.588963    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-istio-podinfo" pod "e00cf9ef-6182-4fc8-b564-ea2425c9e5a5" (UID: "e00cf9ef-6182-4fc8-b564-ea2425c9e5a5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:09:33.589109 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:33.588979    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-kube-api-access-d872l" pod "e00cf9ef-6182-4fc8-b564-ea2425c9e5a5" (UID: "e00cf9ef-6182-4fc8-b564-ea2425c9e5a5"). InnerVolumeSpecName "kube-api-access-d872l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:33.589109 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:33.589079    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-istio-token" pod "e00cf9ef-6182-4fc8-b564-ea2425c9e5a5" (UID: "e00cf9ef-6182-4fc8-b564-ea2425c9e5a5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:33.688202 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:33.688120    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:33.688202 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:33.688158    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:33.688202 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:33.688169    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d872l\" (UniqueName: \"kubernetes.io/projected/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-kube-api-access-d872l\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:33.688202 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:33.688179    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:33.720195 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:33.720159    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:33.890974 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:33.890937    4871 generic.go:356] "Generic (PLEG): container finished" podID="e00cf9ef-6182-4fc8-b564-ea2425c9e5a5" containerID="02d412827d862380b10acbe699f6fcce68156210b81c32b1fa8c2a28c81e62db" exitCode=0
Oct 30 22:09:33.891159 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:33.890991    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"
Oct 30 22:09:33.891159 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:33.891017    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp" event={"ID":"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5","Type":"ContainerDied","Data":"02d412827d862380b10acbe699f6fcce68156210b81c32b1fa8c2a28c81e62db"}
Oct 30 22:09:33.891159 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:33.891073    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp" event={"ID":"e00cf9ef-6182-4fc8-b564-ea2425c9e5a5","Type":"ContainerDied","Data":"5a0fa1c4eb7693da08eece46e0f12ac83b49c9fd40c3bd89d89eb23ed96c78d2"}
Oct 30 22:09:33.891159 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:33.891100    4871 scope.go:122] "RemoveContainer" containerID="02d412827d862380b10acbe699f6fcce68156210b81c32b1fa8c2a28c81e62db"
Oct 30 22:09:33.907469 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:33.907444    4871 scope.go:122] "RemoveContainer" containerID="389157db519b317d7fd8a6435373d87d78a5106c90d871b73e23738271a5ab58"
Oct 30 22:09:33.918936 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:33.918911    4871 scope.go:122] "RemoveContainer" containerID="02d412827d862380b10acbe699f6fcce68156210b81c32b1fa8c2a28c81e62db"
Oct 30 22:09:33.919299 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:09:33.919272    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02d412827d862380b10acbe699f6fcce68156210b81c32b1fa8c2a28c81e62db\": container with ID starting with 02d412827d862380b10acbe699f6fcce68156210b81c32b1fa8c2a28c81e62db not found: ID does not exist" containerID="02d412827d862380b10acbe699f6fcce68156210b81c32b1fa8c2a28c81e62db"
Oct 30 22:09:33.919377 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:33.919313    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02d412827d862380b10acbe699f6fcce68156210b81c32b1fa8c2a28c81e62db"} err="failed to get container status \"02d412827d862380b10acbe699f6fcce68156210b81c32b1fa8c2a28c81e62db\": rpc error: code = NotFound desc = could not find container \"02d412827d862380b10acbe699f6fcce68156210b81c32b1fa8c2a28c81e62db\": container with ID starting with 02d412827d862380b10acbe699f6fcce68156210b81c32b1fa8c2a28c81e62db not found: ID does not exist"
Oct 30 22:09:33.919377 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:33.919342    4871 scope.go:122] "RemoveContainer" containerID="389157db519b317d7fd8a6435373d87d78a5106c90d871b73e23738271a5ab58"
Oct 30 22:09:33.919623 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:09:33.919594    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"389157db519b317d7fd8a6435373d87d78a5106c90d871b73e23738271a5ab58\": container with ID starting with 389157db519b317d7fd8a6435373d87d78a5106c90d871b73e23738271a5ab58 not found: ID does not exist" containerID="389157db519b317d7fd8a6435373d87d78a5106c90d871b73e23738271a5ab58"
Oct 30 22:09:33.919725 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:33.919627    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"389157db519b317d7fd8a6435373d87d78a5106c90d871b73e23738271a5ab58"} err="failed to get container status \"389157db519b317d7fd8a6435373d87d78a5106c90d871b73e23738271a5ab58\": rpc error: code = NotFound desc = could not find container \"389157db519b317d7fd8a6435373d87d78a5106c90d871b73e23738271a5ab58\": container with ID starting with 389157db519b317d7fd8a6435373d87d78a5106c90d871b73e23738271a5ab58 not found: ID does not exist"
Oct 30 22:09:34.050183 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:34.050107    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"]
Oct 30 22:09:34.054167 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:34.054140    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-a5f7b9-openshift-default-99f7c85d7-hzrgp"]
Oct 30 22:09:35.398418 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:35.398347    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"
Oct 30 22:09:35.506522 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:35.506485    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d5e633ba-5844-4f32-956d-8757ce76acce-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5e633ba-5844-4f32-956d-8757ce76acce-credential-socket\") pod \"d5e633ba-5844-4f32-956d-8757ce76acce\" (UID: \"d5e633ba-5844-4f32-956d-8757ce76acce\") "
Oct 30 22:09:35.506691 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:35.506545    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d5e633ba-5844-4f32-956d-8757ce76acce-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5e633ba-5844-4f32-956d-8757ce76acce-workload-certs\") pod \"d5e633ba-5844-4f32-956d-8757ce76acce\" (UID: \"d5e633ba-5844-4f32-956d-8757ce76acce\") "
Oct 30 22:09:35.506691 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:35.506606    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d5e633ba-5844-4f32-956d-8757ce76acce-istio-token\" (UniqueName: \"kubernetes.io/projected/d5e633ba-5844-4f32-956d-8757ce76acce-istio-token\") pod \"d5e633ba-5844-4f32-956d-8757ce76acce\" (UID: \"d5e633ba-5844-4f32-956d-8757ce76acce\") "
Oct 30 22:09:35.506691 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:35.506641    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d5e633ba-5844-4f32-956d-8757ce76acce-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5e633ba-5844-4f32-956d-8757ce76acce-istiod-ca-cert\") pod \"d5e633ba-5844-4f32-956d-8757ce76acce\" (UID: \"d5e633ba-5844-4f32-956d-8757ce76acce\") "
Oct 30 22:09:35.506691 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:35.506673    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d5e633ba-5844-4f32-956d-8757ce76acce-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5e633ba-5844-4f32-956d-8757ce76acce-istio-data\") pod \"d5e633ba-5844-4f32-956d-8757ce76acce\" (UID: \"d5e633ba-5844-4f32-956d-8757ce76acce\") "
Oct 30 22:09:35.506912 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:35.506824    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d5e633ba-5844-4f32-956d-8757ce76acce-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5e633ba-5844-4f32-956d-8757ce76acce-istio-envoy\") pod \"d5e633ba-5844-4f32-956d-8757ce76acce\" (UID: \"d5e633ba-5844-4f32-956d-8757ce76acce\") "
Oct 30 22:09:35.506969 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:35.506907    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5e633ba-5844-4f32-956d-8757ce76acce-workload-certs" pod "d5e633ba-5844-4f32-956d-8757ce76acce" (UID: "d5e633ba-5844-4f32-956d-8757ce76acce"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:35.506969 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:35.506932    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d5e633ba-5844-4f32-956d-8757ce76acce-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5e633ba-5844-4f32-956d-8757ce76acce-workload-socket\") pod \"d5e633ba-5844-4f32-956d-8757ce76acce\" (UID: \"d5e633ba-5844-4f32-956d-8757ce76acce\") "
Oct 30 22:09:35.507095 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:35.506959    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5e633ba-5844-4f32-956d-8757ce76acce-credential-socket" pod "d5e633ba-5844-4f32-956d-8757ce76acce" (UID: "d5e633ba-5844-4f32-956d-8757ce76acce"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:35.507095 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:35.506971    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d5e633ba-5844-4f32-956d-8757ce76acce-kube-api-access-h8nnq\" (UniqueName: \"kubernetes.io/projected/d5e633ba-5844-4f32-956d-8757ce76acce-kube-api-access-h8nnq\") pod \"d5e633ba-5844-4f32-956d-8757ce76acce\" (UID: \"d5e633ba-5844-4f32-956d-8757ce76acce\") "
Oct 30 22:09:35.507203 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:35.507139    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d5e633ba-5844-4f32-956d-8757ce76acce-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5e633ba-5844-4f32-956d-8757ce76acce-istio-podinfo\") pod \"d5e633ba-5844-4f32-956d-8757ce76acce\" (UID: \"d5e633ba-5844-4f32-956d-8757ce76acce\") "
Oct 30 22:09:35.507284 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:35.507194    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5e633ba-5844-4f32-956d-8757ce76acce-istiod-ca-cert" pod "d5e633ba-5844-4f32-956d-8757ce76acce" (UID: "d5e633ba-5844-4f32-956d-8757ce76acce"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:09:35.507284 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:35.507261    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5e633ba-5844-4f32-956d-8757ce76acce-workload-socket" pod "d5e633ba-5844-4f32-956d-8757ce76acce" (UID: "d5e633ba-5844-4f32-956d-8757ce76acce"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:35.507407 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:35.507384    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5e633ba-5844-4f32-956d-8757ce76acce-istio-data" pod "d5e633ba-5844-4f32-956d-8757ce76acce" (UID: "d5e633ba-5844-4f32-956d-8757ce76acce"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:35.507788 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:35.507744    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5e633ba-5844-4f32-956d-8757ce76acce-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:35.507925 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:35.507791    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5e633ba-5844-4f32-956d-8757ce76acce-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:35.507925 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:35.507808    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5e633ba-5844-4f32-956d-8757ce76acce-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:35.507925 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:35.507823    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5e633ba-5844-4f32-956d-8757ce76acce-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:35.507925 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:35.507837    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5e633ba-5844-4f32-956d-8757ce76acce-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:35.510268 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:35.509278    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5e633ba-5844-4f32-956d-8757ce76acce-istio-token" pod "d5e633ba-5844-4f32-956d-8757ce76acce" (UID: "d5e633ba-5844-4f32-956d-8757ce76acce"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:35.510268 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:35.509512    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5e633ba-5844-4f32-956d-8757ce76acce-kube-api-access-h8nnq" pod "d5e633ba-5844-4f32-956d-8757ce76acce" (UID: "d5e633ba-5844-4f32-956d-8757ce76acce"). InnerVolumeSpecName "kube-api-access-h8nnq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:35.510268 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:35.509639    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5e633ba-5844-4f32-956d-8757ce76acce-istio-envoy" pod "d5e633ba-5844-4f32-956d-8757ce76acce" (UID: "d5e633ba-5844-4f32-956d-8757ce76acce"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:35.510268 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:35.509691    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d5e633ba-5844-4f32-956d-8757ce76acce-istio-podinfo" pod "d5e633ba-5844-4f32-956d-8757ce76acce" (UID: "d5e633ba-5844-4f32-956d-8757ce76acce"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:09:35.576494 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:09:35.576450    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:09:35.608566 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:35.608528    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5e633ba-5844-4f32-956d-8757ce76acce-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:35.608566 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:35.608565    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8nnq\" (UniqueName: \"kubernetes.io/projected/d5e633ba-5844-4f32-956d-8757ce76acce-kube-api-access-h8nnq\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:35.608759 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:35.608581    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5e633ba-5844-4f32-956d-8757ce76acce-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:35.608759 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:35.608594    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5e633ba-5844-4f32-956d-8757ce76acce-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:35.912669 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:35.912628    4871 generic.go:356] "Generic (PLEG): container finished" podID="d5e633ba-5844-4f32-956d-8757ce76acce" containerID="f30ddb4ace4b9375945ae44f718c1761e69dfb42111929ccac7ab160d117e158" exitCode=0
Oct 30 22:09:35.912887 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:35.912696    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"
Oct 30 22:09:35.912887 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:35.912827    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l" event={"ID":"d5e633ba-5844-4f32-956d-8757ce76acce","Type":"ContainerDied","Data":"f30ddb4ace4b9375945ae44f718c1761e69dfb42111929ccac7ab160d117e158"}
Oct 30 22:09:35.912887 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:35.912881    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l" event={"ID":"d5e633ba-5844-4f32-956d-8757ce76acce","Type":"ContainerDied","Data":"e95f5103a65d9341c5f5d0a32af18d4c76eb016bb00147413544ecfaa533ee3c"}
Oct 30 22:09:35.913055 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:35.912905    4871 scope.go:122] "RemoveContainer" containerID="f30ddb4ace4b9375945ae44f718c1761e69dfb42111929ccac7ab160d117e158"
Oct 30 22:09:35.924691 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:35.924662    4871 scope.go:122] "RemoveContainer" containerID="be36c755fc6a62139fac349c74f708a0df0de657e27ccd05b45943253d316aae"
Oct 30 22:09:35.934742 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:35.934712    4871 scope.go:122] "RemoveContainer" containerID="f30ddb4ace4b9375945ae44f718c1761e69dfb42111929ccac7ab160d117e158"
Oct 30 22:09:35.935023 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:09:35.935002    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f30ddb4ace4b9375945ae44f718c1761e69dfb42111929ccac7ab160d117e158\": container with ID starting with f30ddb4ace4b9375945ae44f718c1761e69dfb42111929ccac7ab160d117e158 not found: ID does not exist" containerID="f30ddb4ace4b9375945ae44f718c1761e69dfb42111929ccac7ab160d117e158"
Oct 30 22:09:35.935089 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:35.935035    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f30ddb4ace4b9375945ae44f718c1761e69dfb42111929ccac7ab160d117e158"} err="failed to get container status \"f30ddb4ace4b9375945ae44f718c1761e69dfb42111929ccac7ab160d117e158\": rpc error: code = NotFound desc = could not find container \"f30ddb4ace4b9375945ae44f718c1761e69dfb42111929ccac7ab160d117e158\": container with ID starting with f30ddb4ace4b9375945ae44f718c1761e69dfb42111929ccac7ab160d117e158 not found: ID does not exist"
Oct 30 22:09:35.935089 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:35.935054    4871 scope.go:122] "RemoveContainer" containerID="be36c755fc6a62139fac349c74f708a0df0de657e27ccd05b45943253d316aae"
Oct 30 22:09:35.935383 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:09:35.935361    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be36c755fc6a62139fac349c74f708a0df0de657e27ccd05b45943253d316aae\": container with ID starting with be36c755fc6a62139fac349c74f708a0df0de657e27ccd05b45943253d316aae not found: ID does not exist" containerID="be36c755fc6a62139fac349c74f708a0df0de657e27ccd05b45943253d316aae"
Oct 30 22:09:35.935446 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:35.935388    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be36c755fc6a62139fac349c74f708a0df0de657e27ccd05b45943253d316aae"} err="failed to get container status \"be36c755fc6a62139fac349c74f708a0df0de657e27ccd05b45943253d316aae\": rpc error: code = NotFound desc = could not find container \"be36c755fc6a62139fac349c74f708a0df0de657e27ccd05b45943253d316aae\": container with ID starting with be36c755fc6a62139fac349c74f708a0df0de657e27ccd05b45943253d316aae not found: ID does not exist"
Oct 30 22:09:35.962365 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:35.962328    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e00cf9ef-6182-4fc8-b564-ea2425c9e5a5" path="/var/lib/kubelet/pods/e00cf9ef-6182-4fc8-b564-ea2425c9e5a5/volumes"
Oct 30 22:09:36.071282 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:36.071248    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"]
Oct 30 22:09:36.117566 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:36.117530    4871 patch_prober.go:29] interesting pod/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:09:36.117727 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:36.117564    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-4cb675-openshift-default-8685bbd847-v7v7l"]
Oct 30 22:09:36.117727 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:36.117596    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k" podUID="c5b0cd90-fe64-4446-9a53-9ab9ebaa269e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:09:37.659640 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:37.659576    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"
Oct 30 22:09:37.737620 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:37.737583    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-istio-data\") pod \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\" (UID: \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\") "
Oct 30 22:09:37.737793 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:37.737631    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-istiod-ca-cert\") pod \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\" (UID: \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\") "
Oct 30 22:09:37.737793 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:37.737726    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-istio-token\" (UniqueName: \"kubernetes.io/projected/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-istio-token\") pod \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\" (UID: \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\") "
Oct 30 22:09:37.737899 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:37.737819    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-istio-envoy\") pod \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\" (UID: \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\") "
Oct 30 22:09:37.737899 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:37.737852    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-istio-podinfo\") pod \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\" (UID: \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\") "
Oct 30 22:09:37.737899 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:37.737865    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-istio-data" pod "c5b0cd90-fe64-4446-9a53-9ab9ebaa269e" (UID: "c5b0cd90-fe64-4446-9a53-9ab9ebaa269e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:37.738065 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:37.737912    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-workload-socket\") pod \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\" (UID: \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\") "
Oct 30 22:09:37.738065 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:37.738010    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-credential-socket\") pod \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\" (UID: \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\") "
Oct 30 22:09:37.738065 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:37.738032    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-istiod-ca-cert" pod "c5b0cd90-fe64-4446-9a53-9ab9ebaa269e" (UID: "c5b0cd90-fe64-4446-9a53-9ab9ebaa269e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:09:37.738065 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:37.738054    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-workload-certs\") pod \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\" (UID: \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\") "
Oct 30 22:09:37.738288 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:37.738098    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-kube-api-access-d7n99\" (UniqueName: \"kubernetes.io/projected/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-kube-api-access-d7n99\") pod \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\" (UID: \"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e\") "
Oct 30 22:09:37.738288 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:37.738207    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-workload-socket" pod "c5b0cd90-fe64-4446-9a53-9ab9ebaa269e" (UID: "c5b0cd90-fe64-4446-9a53-9ab9ebaa269e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:37.738412 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:37.738389    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-credential-socket" pod "c5b0cd90-fe64-4446-9a53-9ab9ebaa269e" (UID: "c5b0cd90-fe64-4446-9a53-9ab9ebaa269e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:37.738612 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:37.738577    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-workload-certs" pod "c5b0cd90-fe64-4446-9a53-9ab9ebaa269e" (UID: "c5b0cd90-fe64-4446-9a53-9ab9ebaa269e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:37.738708 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:37.738684    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:37.738708 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:37.738704    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:37.738828 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:37.738718    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:37.738828 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:37.738733    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:37.738828 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:37.738745    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:37.740659 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:37.740616    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-istio-envoy" pod "c5b0cd90-fe64-4446-9a53-9ab9ebaa269e" (UID: "c5b0cd90-fe64-4446-9a53-9ab9ebaa269e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:37.740749 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:37.740646    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-istio-podinfo" pod "c5b0cd90-fe64-4446-9a53-9ab9ebaa269e" (UID: "c5b0cd90-fe64-4446-9a53-9ab9ebaa269e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:09:37.740749 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:37.740722    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-kube-api-access-d7n99" pod "c5b0cd90-fe64-4446-9a53-9ab9ebaa269e" (UID: "c5b0cd90-fe64-4446-9a53-9ab9ebaa269e"). InnerVolumeSpecName "kube-api-access-d7n99". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:37.740830 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:37.740750    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-istio-token" pod "c5b0cd90-fe64-4446-9a53-9ab9ebaa269e" (UID: "c5b0cd90-fe64-4446-9a53-9ab9ebaa269e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:37.800108 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:37.800067    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:37.844267 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:37.840224    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d7n99\" (UniqueName: \"kubernetes.io/projected/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-kube-api-access-d7n99\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:37.844267 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:37.840369    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:37.844267 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:37.840394    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:37.844267 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:37.840408    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:37.933285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:37.933163    4871 generic.go:356] "Generic (PLEG): container finished" podID="c5b0cd90-fe64-4446-9a53-9ab9ebaa269e" containerID="5684f813bd9bb13ddba214ca358f59f5f40f290556249dd8396053a96cba135f" exitCode=0
Oct 30 22:09:37.933285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:37.933212    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"
Oct 30 22:09:37.933504 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:37.933276    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k" event={"ID":"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e","Type":"ContainerDied","Data":"5684f813bd9bb13ddba214ca358f59f5f40f290556249dd8396053a96cba135f"}
Oct 30 22:09:37.933504 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:37.933334    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k" event={"ID":"c5b0cd90-fe64-4446-9a53-9ab9ebaa269e","Type":"ContainerDied","Data":"9b5e481e1c830e69b97a9ef42a9261ebeaa2131940f78961eb4b416dde5e13c4"}
Oct 30 22:09:37.933504 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:37.933362    4871 scope.go:122] "RemoveContainer" containerID="5684f813bd9bb13ddba214ca358f59f5f40f290556249dd8396053a96cba135f"
Oct 30 22:09:37.945191 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:37.945169    4871 scope.go:122] "RemoveContainer" containerID="5684f813bd9bb13ddba214ca358f59f5f40f290556249dd8396053a96cba135f"
Oct 30 22:09:37.945515 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:09:37.945494    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5684f813bd9bb13ddba214ca358f59f5f40f290556249dd8396053a96cba135f\": container with ID starting with 5684f813bd9bb13ddba214ca358f59f5f40f290556249dd8396053a96cba135f not found: ID does not exist" containerID="5684f813bd9bb13ddba214ca358f59f5f40f290556249dd8396053a96cba135f"
Oct 30 22:09:37.945574 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:37.945524    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5684f813bd9bb13ddba214ca358f59f5f40f290556249dd8396053a96cba135f"} err="failed to get container status \"5684f813bd9bb13ddba214ca358f59f5f40f290556249dd8396053a96cba135f\": rpc error: code = NotFound desc = could not find container \"5684f813bd9bb13ddba214ca358f59f5f40f290556249dd8396053a96cba135f\": container with ID starting with 5684f813bd9bb13ddba214ca358f59f5f40f290556249dd8396053a96cba135f not found: ID does not exist"
Oct 30 22:09:37.960162 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:37.960132    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d5e633ba-5844-4f32-956d-8757ce76acce" path="/var/lib/kubelet/pods/d5e633ba-5844-4f32-956d-8757ce76acce/volumes"
Oct 30 22:09:38.099385 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:38.099345    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"]
Oct 30 22:09:38.106381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:38.106348    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-wvz9k"]
Oct 30 22:09:39.287570 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:39.287533    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:39.961409 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:39.961374    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c5b0cd90-fe64-4446-9a53-9ab9ebaa269e" path="/var/lib/kubelet/pods/c5b0cd90-fe64-4446-9a53-9ab9ebaa269e/volumes"
Oct 30 22:09:40.027652 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:40.027595    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:40.198430 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:40.198394    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:40.389911 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:40.389873    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"]
Oct 30 22:09:40.390432 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:40.390150    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4" podUID="722963e5-d33d-42a6-9f76-7c9b8d2b2fe0" containerName="istio-proxy" containerID="cri-o://9c0c58dc21db3aa8e2a7a12aba5cf75b7672a1edb6f583cefcd5823cb2abef8c" gracePeriod=30
Oct 30 22:09:40.407138 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:40.407102    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:42.749474 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:42.749430    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:45.680647 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:45.680617    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"
Oct 30 22:09:45.739601 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:45.739556    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-workload-certs\") pod \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\" (UID: \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\") "
Oct 30 22:09:45.739601 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:45.739601    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-istiod-ca-cert\") pod \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\" (UID: \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\") "
Oct 30 22:09:45.739880 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:45.739645    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-istio-data\") pod \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\" (UID: \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\") "
Oct 30 22:09:45.739880 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:45.739683    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-istio-envoy\") pod \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\" (UID: \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\") "
Oct 30 22:09:45.739880 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:45.739715    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-istio-token\" (UniqueName: \"kubernetes.io/projected/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-istio-token\") pod \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\" (UID: \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\") "
Oct 30 22:09:45.739880 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:45.739760    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-kube-api-access-jczbn\" (UniqueName: \"kubernetes.io/projected/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-kube-api-access-jczbn\") pod \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\" (UID: \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\") "
Oct 30 22:09:45.739880 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:45.739789    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-workload-socket\") pod \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\" (UID: \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\") "
Oct 30 22:09:45.739880 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:45.739869    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-credential-socket\") pod \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\" (UID: \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\") "
Oct 30 22:09:45.740141 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:45.739965    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-istio-podinfo\") pod \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\" (UID: \"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0\") "
Oct 30 22:09:45.740186 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:45.740146    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-workload-certs" pod "722963e5-d33d-42a6-9f76-7c9b8d2b2fe0" (UID: "722963e5-d33d-42a6-9f76-7c9b8d2b2fe0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:45.740320 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:45.740293    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-istio-data" pod "722963e5-d33d-42a6-9f76-7c9b8d2b2fe0" (UID: "722963e5-d33d-42a6-9f76-7c9b8d2b2fe0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:45.740417 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:45.740393    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-istiod-ca-cert" pod "722963e5-d33d-42a6-9f76-7c9b8d2b2fe0" (UID: "722963e5-d33d-42a6-9f76-7c9b8d2b2fe0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:09:45.740466 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:45.740415    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-credential-socket" pod "722963e5-d33d-42a6-9f76-7c9b8d2b2fe0" (UID: "722963e5-d33d-42a6-9f76-7c9b8d2b2fe0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:45.740661 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:45.740638    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-workload-socket" pod "722963e5-d33d-42a6-9f76-7c9b8d2b2fe0" (UID: "722963e5-d33d-42a6-9f76-7c9b8d2b2fe0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:45.740797 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:45.740677    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:45.740797 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:45.740749    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:45.740797 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:45.740766    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:45.740965 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:45.740795    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:45.742634 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:45.742608    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-istio-podinfo" pod "722963e5-d33d-42a6-9f76-7c9b8d2b2fe0" (UID: "722963e5-d33d-42a6-9f76-7c9b8d2b2fe0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:09:45.742817 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:45.742792    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-istio-envoy" pod "722963e5-d33d-42a6-9f76-7c9b8d2b2fe0" (UID: "722963e5-d33d-42a6-9f76-7c9b8d2b2fe0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:45.742917 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:45.742838    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-kube-api-access-jczbn" pod "722963e5-d33d-42a6-9f76-7c9b8d2b2fe0" (UID: "722963e5-d33d-42a6-9f76-7c9b8d2b2fe0"). InnerVolumeSpecName "kube-api-access-jczbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:45.742917 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:45.742866    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-istio-token" pod "722963e5-d33d-42a6-9f76-7c9b8d2b2fe0" (UID: "722963e5-d33d-42a6-9f76-7c9b8d2b2fe0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:45.841996 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:45.841956    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:45.841996 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:45.841989    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:45.841996 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:45.841999    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:45.842273 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:45.842010    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jczbn\" (UniqueName: \"kubernetes.io/projected/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-kube-api-access-jczbn\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:45.842273 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:45.842029    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:09:45.998055 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:45.998014    4871 generic.go:356] "Generic (PLEG): container finished" podID="722963e5-d33d-42a6-9f76-7c9b8d2b2fe0" containerID="9c0c58dc21db3aa8e2a7a12aba5cf75b7672a1edb6f583cefcd5823cb2abef8c" exitCode=0
Oct 30 22:09:45.998221 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:45.998067    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"
Oct 30 22:09:45.998221 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:45.998101    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4" event={"ID":"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0","Type":"ContainerDied","Data":"9c0c58dc21db3aa8e2a7a12aba5cf75b7672a1edb6f583cefcd5823cb2abef8c"}
Oct 30 22:09:45.998221 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:45.998159    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4" event={"ID":"722963e5-d33d-42a6-9f76-7c9b8d2b2fe0","Type":"ContainerDied","Data":"87a102e4b8e5a6a5e5291600a5dc786ab35b050e9ed0bda4e289ded1aaed2ac4"}
Oct 30 22:09:45.998221 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:45.998184    4871 scope.go:122] "RemoveContainer" containerID="9c0c58dc21db3aa8e2a7a12aba5cf75b7672a1edb6f583cefcd5823cb2abef8c"
Oct 30 22:09:46.009090 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:46.009069    4871 scope.go:122] "RemoveContainer" containerID="96f88fa306b5954d40cf5ef622ef33aadbe249cda9dbd69cb5452f869563ba45"
Oct 30 22:09:46.021432 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:46.021409    4871 scope.go:122] "RemoveContainer" containerID="9c0c58dc21db3aa8e2a7a12aba5cf75b7672a1edb6f583cefcd5823cb2abef8c"
Oct 30 22:09:46.021738 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:09:46.021719    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c0c58dc21db3aa8e2a7a12aba5cf75b7672a1edb6f583cefcd5823cb2abef8c\": container with ID starting with 9c0c58dc21db3aa8e2a7a12aba5cf75b7672a1edb6f583cefcd5823cb2abef8c not found: ID does not exist" containerID="9c0c58dc21db3aa8e2a7a12aba5cf75b7672a1edb6f583cefcd5823cb2abef8c"
Oct 30 22:09:46.021791 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:46.021748    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c0c58dc21db3aa8e2a7a12aba5cf75b7672a1edb6f583cefcd5823cb2abef8c"} err="failed to get container status \"9c0c58dc21db3aa8e2a7a12aba5cf75b7672a1edb6f583cefcd5823cb2abef8c\": rpc error: code = NotFound desc = could not find container \"9c0c58dc21db3aa8e2a7a12aba5cf75b7672a1edb6f583cefcd5823cb2abef8c\": container with ID starting with 9c0c58dc21db3aa8e2a7a12aba5cf75b7672a1edb6f583cefcd5823cb2abef8c not found: ID does not exist"
Oct 30 22:09:46.021791 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:46.021766    4871 scope.go:122] "RemoveContainer" containerID="96f88fa306b5954d40cf5ef622ef33aadbe249cda9dbd69cb5452f869563ba45"
Oct 30 22:09:46.022114 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:09:46.022089    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96f88fa306b5954d40cf5ef622ef33aadbe249cda9dbd69cb5452f869563ba45\": container with ID starting with 96f88fa306b5954d40cf5ef622ef33aadbe249cda9dbd69cb5452f869563ba45 not found: ID does not exist" containerID="96f88fa306b5954d40cf5ef622ef33aadbe249cda9dbd69cb5452f869563ba45"
Oct 30 22:09:46.022175 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:46.022124    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96f88fa306b5954d40cf5ef622ef33aadbe249cda9dbd69cb5452f869563ba45"} err="failed to get container status \"96f88fa306b5954d40cf5ef622ef33aadbe249cda9dbd69cb5452f869563ba45\": rpc error: code = NotFound desc = could not find container \"96f88fa306b5954d40cf5ef622ef33aadbe249cda9dbd69cb5452f869563ba45\": container with ID starting with 96f88fa306b5954d40cf5ef622ef33aadbe249cda9dbd69cb5452f869563ba45 not found: ID does not exist"
Oct 30 22:09:46.156196 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:46.156156    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"]
Oct 30 22:09:46.160587 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:46.160552    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-82b62e-openshift-default-7dcfbcf48f-mqpm4"]
Oct 30 22:09:47.580615 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:47.580579    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:47.961051 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:47.960947    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="722963e5-d33d-42a6-9f76-7c9b8d2b2fe0" path="/var/lib/kubelet/pods/722963e5-d33d-42a6-9f76-7c9b8d2b2fe0/volumes"
Oct 30 22:09:51.887305 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.887268    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz"]
Oct 30 22:09:51.888122 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.888100    4871 cpu_manager.go:414] "containerMap: removing container" podUID="e00cf9ef-6182-4fc8-b564-ea2425c9e5a5" containerName="istio-proxy"
Oct 30 22:09:51.888193 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.888125    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e00cf9ef-6182-4fc8-b564-ea2425c9e5a5" containerName="istio-proxy"
Oct 30 22:09:51.888193 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.888145    4871 cpu_manager.go:414] "containerMap: removing container" podUID="f0cbb7ad-f150-4562-bb4c-ad9dce4545bd" containerName="istio-proxy"
Oct 30 22:09:51.888193 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.888153    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f0cbb7ad-f150-4562-bb4c-ad9dce4545bd" containerName="istio-proxy"
Oct 30 22:09:51.888193 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.888179    4871 cpu_manager.go:414] "containerMap: removing container" podUID="d5e633ba-5844-4f32-956d-8757ce76acce" containerName="istio-proxy"
Oct 30 22:09:51.888193 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.888189    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d5e633ba-5844-4f32-956d-8757ce76acce" containerName="istio-proxy"
Oct 30 22:09:51.888371 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.888201    4871 cpu_manager.go:414] "containerMap: removing container" podUID="0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd" containerName="maas-api"
Oct 30 22:09:51.888371 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.888209    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd" containerName="maas-api"
Oct 30 22:09:51.888371 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.888221    4871 cpu_manager.go:414] "containerMap: removing container" podUID="198884dd-8c10-4d1f-9f64-6e00c0e7b6ab" containerName="istio-proxy"
Oct 30 22:09:51.888371 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.888228    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="198884dd-8c10-4d1f-9f64-6e00c0e7b6ab" containerName="istio-proxy"
Oct 30 22:09:51.888371 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.888264    4871 cpu_manager.go:414] "containerMap: removing container" podUID="904ace00-01d5-4e6b-ba97-d9e887b5da40" containerName="maas-api"
Oct 30 22:09:51.888371 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.888271    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="904ace00-01d5-4e6b-ba97-d9e887b5da40" containerName="maas-api"
Oct 30 22:09:51.888371 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.888298    4871 cpu_manager.go:414] "containerMap: removing container" podUID="a83867ab-e545-4023-8923-fc0124c9c638" containerName="istio-proxy"
Oct 30 22:09:51.888371 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.888305    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a83867ab-e545-4023-8923-fc0124c9c638" containerName="istio-proxy"
Oct 30 22:09:51.888371 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.888318    4871 cpu_manager.go:414] "containerMap: removing container" podUID="d5e633ba-5844-4f32-956d-8757ce76acce" containerName="istio-proxy"
Oct 30 22:09:51.888371 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.888325    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d5e633ba-5844-4f32-956d-8757ce76acce" containerName="istio-proxy"
Oct 30 22:09:51.888371 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.888342    4871 cpu_manager.go:414] "containerMap: removing container" podUID="c5b0cd90-fe64-4446-9a53-9ab9ebaa269e" containerName="istio-proxy"
Oct 30 22:09:51.888371 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.888348    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c5b0cd90-fe64-4446-9a53-9ab9ebaa269e" containerName="istio-proxy"
Oct 30 22:09:51.888371 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.888364    4871 cpu_manager.go:414] "containerMap: removing container" podUID="722963e5-d33d-42a6-9f76-7c9b8d2b2fe0" containerName="istio-proxy"
Oct 30 22:09:51.888371 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.888371    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="722963e5-d33d-42a6-9f76-7c9b8d2b2fe0" containerName="istio-proxy"
Oct 30 22:09:51.888824 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.888389    4871 cpu_manager.go:414] "containerMap: removing container" podUID="957af2a6-bd01-49ed-a36a-1844daeebf2b" containerName="istio-proxy"
Oct 30 22:09:51.888824 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.888396    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="957af2a6-bd01-49ed-a36a-1844daeebf2b" containerName="istio-proxy"
Oct 30 22:09:51.888824 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.888407    4871 cpu_manager.go:414] "containerMap: removing container" podUID="722963e5-d33d-42a6-9f76-7c9b8d2b2fe0" containerName="istio-proxy"
Oct 30 22:09:51.888824 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.888414    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="722963e5-d33d-42a6-9f76-7c9b8d2b2fe0" containerName="istio-proxy"
Oct 30 22:09:51.888824 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.888524    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="e00cf9ef-6182-4fc8-b564-ea2425c9e5a5" containerName="istio-proxy"
Oct 30 22:09:51.888824 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.888532    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="a83867ab-e545-4023-8923-fc0124c9c638" containerName="istio-proxy"
Oct 30 22:09:51.888824 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.888541    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="d5e633ba-5844-4f32-956d-8757ce76acce" containerName="istio-proxy"
Oct 30 22:09:51.888824 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.888548    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="c5b0cd90-fe64-4446-9a53-9ab9ebaa269e" containerName="istio-proxy"
Oct 30 22:09:51.888824 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.888555    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="ef03dee1-e693-42a0-ace8-4a28e099b92b" containerName="istio-proxy"
Oct 30 22:09:51.888824 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.888561    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="722963e5-d33d-42a6-9f76-7c9b8d2b2fe0" containerName="istio-proxy"
Oct 30 22:09:51.888824 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.888567    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="957af2a6-bd01-49ed-a36a-1844daeebf2b" containerName="istio-proxy"
Oct 30 22:09:51.888824 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.888573    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="722963e5-d33d-42a6-9f76-7c9b8d2b2fe0" containerName="istio-proxy"
Oct 30 22:09:51.888824 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.888579    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="e00cf9ef-6182-4fc8-b564-ea2425c9e5a5" containerName="istio-proxy"
Oct 30 22:09:51.888824 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.888586    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="d5e633ba-5844-4f32-956d-8757ce76acce" containerName="istio-proxy"
Oct 30 22:09:51.888824 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.888592    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="48a6bcac-9719-4358-9019-71fd0a388ba1" containerName="istio-proxy"
Oct 30 22:09:51.888824 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.888598    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="0fce1e8b-09a8-4ba9-bc17-ba70a29f8ecd" containerName="maas-api"
Oct 30 22:09:51.888824 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.888604    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="198884dd-8c10-4d1f-9f64-6e00c0e7b6ab" containerName="istio-proxy"
Oct 30 22:09:51.888824 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.888610    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="198884dd-8c10-4d1f-9f64-6e00c0e7b6ab" containerName="istio-proxy"
Oct 30 22:09:51.888824 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.888616    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="904ace00-01d5-4e6b-ba97-d9e887b5da40" containerName="maas-api"
Oct 30 22:09:51.893247 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.893212    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz"
Oct 30 22:09:51.947480 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.947440    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz"]
Oct 30 22:09:51.988160 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:51.988122    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-52f191-serving-cert\""
Oct 30 22:09:52.024281 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:52.023198    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb723e66-473c-407f-ad2f-bbd77f86a6bf-metrics-certs\") pod \"maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz\" (UID: \"eb723e66-473c-407f-ad2f-bbd77f86a6bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz"
Oct 30 22:09:52.024281 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:52.023409    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n98sn\" (UniqueName: \"kubernetes.io/projected/eb723e66-473c-407f-ad2f-bbd77f86a6bf-kube-api-access-n98sn\") pod \"maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz\" (UID: \"eb723e66-473c-407f-ad2f-bbd77f86a6bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz"
Oct 30 22:09:52.024281 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:52.023548    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb723e66-473c-407f-ad2f-bbd77f86a6bf-maas-api-tls\") pod \"maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz\" (UID: \"eb723e66-473c-407f-ad2f-bbd77f86a6bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz"
Oct 30 22:09:52.024281 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:52.023626    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb723e66-473c-407f-ad2f-bbd77f86a6bf-openshift-service-ca\") pod \"maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz\" (UID: \"eb723e66-473c-407f-ad2f-bbd77f86a6bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz"
Oct 30 22:09:52.079254 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:52.079216    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:52.124777 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:52.124729    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb723e66-473c-407f-ad2f-bbd77f86a6bf-metrics-certs\") pod \"maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz\" (UID: \"eb723e66-473c-407f-ad2f-bbd77f86a6bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz"
Oct 30 22:09:52.124993 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:52.124878    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n98sn\" (UniqueName: \"kubernetes.io/projected/eb723e66-473c-407f-ad2f-bbd77f86a6bf-kube-api-access-n98sn\") pod \"maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz\" (UID: \"eb723e66-473c-407f-ad2f-bbd77f86a6bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz"
Oct 30 22:09:52.125076 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:52.124989    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb723e66-473c-407f-ad2f-bbd77f86a6bf-maas-api-tls\") pod \"maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz\" (UID: \"eb723e66-473c-407f-ad2f-bbd77f86a6bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz"
Oct 30 22:09:52.125135 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:52.125093    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb723e66-473c-407f-ad2f-bbd77f86a6bf-openshift-service-ca\") pod \"maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz\" (UID: \"eb723e66-473c-407f-ad2f-bbd77f86a6bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz"
Oct 30 22:09:52.126069 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:52.126041    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb723e66-473c-407f-ad2f-bbd77f86a6bf-openshift-service-ca\") pod \"maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz\" (UID: \"eb723e66-473c-407f-ad2f-bbd77f86a6bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz"
Oct 30 22:09:52.127585 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:52.127555    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb723e66-473c-407f-ad2f-bbd77f86a6bf-maas-api-tls\") pod \"maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz\" (UID: \"eb723e66-473c-407f-ad2f-bbd77f86a6bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz"
Oct 30 22:09:52.127665 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:52.127653    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb723e66-473c-407f-ad2f-bbd77f86a6bf-metrics-certs\") pod \"maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz\" (UID: \"eb723e66-473c-407f-ad2f-bbd77f86a6bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz"
Oct 30 22:09:52.183638 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:52.183560    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n98sn\" (UniqueName: \"kubernetes.io/projected/eb723e66-473c-407f-ad2f-bbd77f86a6bf-kube-api-access-n98sn\") pod \"maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz\" (UID: \"eb723e66-473c-407f-ad2f-bbd77f86a6bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz"
Oct 30 22:09:52.208481 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:52.208449    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz"
Oct 30 22:09:52.252021 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:52.251991    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:52.633887 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:52.633853    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz"]
Oct 30 22:09:52.635124 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:09:52.635099    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb723e66_473c_407f_ad2f_bbd77f86a6bf.slice/crio-a667d62e0d323e14e2d3e4049262be51d4d3a46418505cd2d0d39ab9d0e37bdc WatchSource:0}: Error finding container a667d62e0d323e14e2d3e4049262be51d4d3a46418505cd2d0d39ab9d0e37bdc: Status 404 returned error can't find the container with id a667d62e0d323e14e2d3e4049262be51d4d3a46418505cd2d0d39ab9d0e37bdc
Oct 30 22:09:52.636613 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:52.636592    4871 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:09:53.051907 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:53.051810    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz" event={"ID":"eb723e66-473c-407f-ad2f-bbd77f86a6bf","Type":"ContainerStarted","Data":"a667d62e0d323e14e2d3e4049262be51d4d3a46418505cd2d0d39ab9d0e37bdc"}
Oct 30 22:09:54.069721 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:54.069622    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz" event={"ID":"eb723e66-473c-407f-ad2f-bbd77f86a6bf","Type":"ContainerStarted","Data":"8ceee39c5a91ebbb6902a0f59aca868021c4df57a05eeb6f1cb2f342a684103a"}
Oct 30 22:09:54.070190 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:54.069885    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz"
Oct 30 22:09:54.182610 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:54.182547    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz" podStartSLOduration=2.181552757 podStartE2EDuration="3.182532917s" totalImagesPullingTime="1.00098016s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:09:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:09:54.181363224 +0000 UTC m=+1936.358851841" watchObservedRunningTime="2026-10-30 22:09:54.182532917 +0000 UTC m=+1936.360021573"
Oct 30 22:09:55.300802 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:55.300765    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:55.455134 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:55.455096    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:56.843799 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:09:56.843763    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:00.082704 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:00.082676    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz"
Oct 30 22:10:01.410808 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:01.410770    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:02.619360 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:02.619314    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:03.479172 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:03.479134    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"]
Oct 30 22:10:03.479930 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:03.479892    4871 cpu_manager.go:414] "containerMap: removing container" podUID="e00cf9ef-6182-4fc8-b564-ea2425c9e5a5" containerName="istio-proxy"
Oct 30 22:10:03.479930 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:03.479919    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e00cf9ef-6182-4fc8-b564-ea2425c9e5a5" containerName="istio-proxy"
Oct 30 22:10:03.480101 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:03.479956    4871 cpu_manager.go:414] "containerMap: removing container" podUID="198884dd-8c10-4d1f-9f64-6e00c0e7b6ab" containerName="istio-proxy"
Oct 30 22:10:03.480101 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:03.479965    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="198884dd-8c10-4d1f-9f64-6e00c0e7b6ab" containerName="istio-proxy"
Oct 30 22:10:03.484036 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:03.484017    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"
Oct 30 22:10:03.534127 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:03.534083    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-f0b8c9-openshift-default-dockercfg-2kpvm\""
Oct 30 22:10:03.581270 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:03.581206    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:03.584633 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:03.584601    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"]
Oct 30 22:10:03.639511 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:03.639467    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22d76305-7ecb-421f-9926-b1de7d91d2d8-workload-socket\") pod \"e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj\" (UID: \"22d76305-7ecb-421f-9926-b1de7d91d2d8\") " pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"
Oct 30 22:10:03.652764 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:03.652712    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22d76305-7ecb-421f-9926-b1de7d91d2d8-workload-certs\") pod \"e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj\" (UID: \"22d76305-7ecb-421f-9926-b1de7d91d2d8\") " pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"
Oct 30 22:10:03.653000 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:03.652980    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nk7dm\" (UniqueName: \"kubernetes.io/projected/22d76305-7ecb-421f-9926-b1de7d91d2d8-kube-api-access-nk7dm\") pod \"e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj\" (UID: \"22d76305-7ecb-421f-9926-b1de7d91d2d8\") " pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"
Oct 30 22:10:03.653140 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:03.653125    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22d76305-7ecb-421f-9926-b1de7d91d2d8-istio-envoy\") pod \"e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj\" (UID: \"22d76305-7ecb-421f-9926-b1de7d91d2d8\") " pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"
Oct 30 22:10:03.653286 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:03.653269    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22d76305-7ecb-421f-9926-b1de7d91d2d8-istio-token\") pod \"e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj\" (UID: \"22d76305-7ecb-421f-9926-b1de7d91d2d8\") " pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"
Oct 30 22:10:03.657496 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:03.657432    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22d76305-7ecb-421f-9926-b1de7d91d2d8-istio-data\") pod \"e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj\" (UID: \"22d76305-7ecb-421f-9926-b1de7d91d2d8\") " pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"
Oct 30 22:10:03.657660 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:03.657539    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22d76305-7ecb-421f-9926-b1de7d91d2d8-istiod-ca-cert\") pod \"e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj\" (UID: \"22d76305-7ecb-421f-9926-b1de7d91d2d8\") " pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"
Oct 30 22:10:03.657660 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:03.657630    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22d76305-7ecb-421f-9926-b1de7d91d2d8-credential-socket\") pod \"e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj\" (UID: \"22d76305-7ecb-421f-9926-b1de7d91d2d8\") " pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"
Oct 30 22:10:03.657786 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:03.657712    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22d76305-7ecb-421f-9926-b1de7d91d2d8-istio-podinfo\") pod \"e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj\" (UID: \"22d76305-7ecb-421f-9926-b1de7d91d2d8\") " pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"
Oct 30 22:10:03.762317 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:03.758937    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nk7dm\" (UniqueName: \"kubernetes.io/projected/22d76305-7ecb-421f-9926-b1de7d91d2d8-kube-api-access-nk7dm\") pod \"e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj\" (UID: \"22d76305-7ecb-421f-9926-b1de7d91d2d8\") " pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"
Oct 30 22:10:03.762317 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:03.759008    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22d76305-7ecb-421f-9926-b1de7d91d2d8-istio-envoy\") pod \"e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj\" (UID: \"22d76305-7ecb-421f-9926-b1de7d91d2d8\") " pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"
Oct 30 22:10:03.762317 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:03.759058    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22d76305-7ecb-421f-9926-b1de7d91d2d8-istio-token\") pod \"e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj\" (UID: \"22d76305-7ecb-421f-9926-b1de7d91d2d8\") " pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"
Oct 30 22:10:03.762317 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:03.759099    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22d76305-7ecb-421f-9926-b1de7d91d2d8-istio-data\") pod \"e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj\" (UID: \"22d76305-7ecb-421f-9926-b1de7d91d2d8\") " pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"
Oct 30 22:10:03.762317 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:03.759159    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22d76305-7ecb-421f-9926-b1de7d91d2d8-istiod-ca-cert\") pod \"e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj\" (UID: \"22d76305-7ecb-421f-9926-b1de7d91d2d8\") " pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"
Oct 30 22:10:03.762317 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:03.759255    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22d76305-7ecb-421f-9926-b1de7d91d2d8-credential-socket\") pod \"e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj\" (UID: \"22d76305-7ecb-421f-9926-b1de7d91d2d8\") " pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"
Oct 30 22:10:03.762317 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:03.759355    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22d76305-7ecb-421f-9926-b1de7d91d2d8-istio-podinfo\") pod \"e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj\" (UID: \"22d76305-7ecb-421f-9926-b1de7d91d2d8\") " pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"
Oct 30 22:10:03.762317 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:03.759407    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22d76305-7ecb-421f-9926-b1de7d91d2d8-workload-socket\") pod \"e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj\" (UID: \"22d76305-7ecb-421f-9926-b1de7d91d2d8\") " pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"
Oct 30 22:10:03.762317 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:03.759444    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22d76305-7ecb-421f-9926-b1de7d91d2d8-workload-certs\") pod \"e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj\" (UID: \"22d76305-7ecb-421f-9926-b1de7d91d2d8\") " pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"
Oct 30 22:10:03.762317 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:03.759918    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22d76305-7ecb-421f-9926-b1de7d91d2d8-workload-certs\") pod \"e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj\" (UID: \"22d76305-7ecb-421f-9926-b1de7d91d2d8\") " pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"
Oct 30 22:10:03.764092 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:03.764062    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22d76305-7ecb-421f-9926-b1de7d91d2d8-istiod-ca-cert\") pod \"e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj\" (UID: \"22d76305-7ecb-421f-9926-b1de7d91d2d8\") " pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"
Oct 30 22:10:03.764951 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:03.764557    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22d76305-7ecb-421f-9926-b1de7d91d2d8-credential-socket\") pod \"e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj\" (UID: \"22d76305-7ecb-421f-9926-b1de7d91d2d8\") " pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"
Oct 30 22:10:03.765165 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:03.764811    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22d76305-7ecb-421f-9926-b1de7d91d2d8-istio-data\") pod \"e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj\" (UID: \"22d76305-7ecb-421f-9926-b1de7d91d2d8\") " pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"
Oct 30 22:10:03.768527 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:03.768461    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22d76305-7ecb-421f-9926-b1de7d91d2d8-istio-podinfo\") pod \"e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj\" (UID: \"22d76305-7ecb-421f-9926-b1de7d91d2d8\") " pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"
Oct 30 22:10:03.769209 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:03.769187    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22d76305-7ecb-421f-9926-b1de7d91d2d8-istio-envoy\") pod \"e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj\" (UID: \"22d76305-7ecb-421f-9926-b1de7d91d2d8\") " pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"
Oct 30 22:10:03.769445 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:03.764230    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22d76305-7ecb-421f-9926-b1de7d91d2d8-workload-socket\") pod \"e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj\" (UID: \"22d76305-7ecb-421f-9926-b1de7d91d2d8\") " pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"
Oct 30 22:10:03.790770 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:03.790722    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:03.821542 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:03.821508    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22d76305-7ecb-421f-9926-b1de7d91d2d8-istio-token\") pod \"e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj\" (UID: \"22d76305-7ecb-421f-9926-b1de7d91d2d8\") " pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"
Oct 30 22:10:03.822218 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:03.822196    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nk7dm\" (UniqueName: \"kubernetes.io/projected/22d76305-7ecb-421f-9926-b1de7d91d2d8-kube-api-access-nk7dm\") pod \"e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj\" (UID: \"22d76305-7ecb-421f-9926-b1de7d91d2d8\") " pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"
Oct 30 22:10:04.099416 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:04.099312    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"
Oct 30 22:10:04.529796 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:10:04.529752    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22d76305_7ecb_421f_9926_b1de7d91d2d8.slice/crio-9c44e5e1193952642b5a183b4325c7c925c04f9708ef241ba1a3b21214f6fe6b WatchSource:0}: Error finding container 9c44e5e1193952642b5a183b4325c7c925c04f9708ef241ba1a3b21214f6fe6b: Status 404 returned error can't find the container with id 9c44e5e1193952642b5a183b4325c7c925c04f9708ef241ba1a3b21214f6fe6b
Oct 30 22:10:04.530169 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:04.530142    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"]
Oct 30 22:10:04.532421 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:04.532388    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:10:04.532533 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:04.532453    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:10:04.532533 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:04.532483    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:10:05.155432 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:05.155387    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj" event={"ID":"22d76305-7ecb-421f-9926-b1de7d91d2d8","Type":"ContainerStarted","Data":"5a883d0c34ef836dd4d3ec482f188166cbd34fdfaf1dbb97f50608ffff56f88d"}
Oct 30 22:10:05.155850 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:05.155490    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj" event={"ID":"22d76305-7ecb-421f-9926-b1de7d91d2d8","Type":"ContainerStarted","Data":"9c44e5e1193952642b5a183b4325c7c925c04f9708ef241ba1a3b21214f6fe6b"}
Oct 30 22:10:05.367398 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:05.367336    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj" podStartSLOduration=2.367319631 podStartE2EDuration="2.367319631s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:10:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:10:05.31655978 +0000 UTC m=+1947.494048397" watchObservedRunningTime="2026-10-30 22:10:05.367319631 +0000 UTC m=+1947.544808247"
Oct 30 22:10:05.568695 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:10:05.568607    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:10:06.099828 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:06.099798    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"
Oct 30 22:10:06.105819 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:06.105781    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"
Oct 30 22:10:06.164000 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:06.163967    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"
Oct 30 22:10:06.164908 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:06.164890    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"
Oct 30 22:10:06.200959 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:06.200923    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:10.803381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:10.803345    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:14.755337 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:14.755297    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-97f097-85dddff48-dj9xs"]
Oct 30 22:10:14.755820 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:14.755561    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-97f097-85dddff48-dj9xs" podUID="95aa3a5c-4534-4ca7-a4ca-f3db06d36633" containerName="maas-api" containerID="cri-o://496f0bb133b41c2c10cbdebd5fa4f5a97c12970bd8d369ffb638e13feed1858e" gracePeriod=30
Oct 30 22:10:14.860519 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:14.860479    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:15.101499 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:15.099699    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-97f097-85dddff48-dj9xs"
Oct 30 22:10:15.248501 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:15.248461    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/95aa3a5c-4534-4ca7-a4ca-f3db06d36633-kube-api-access-ls64x\" (UniqueName: \"kubernetes.io/projected/95aa3a5c-4534-4ca7-a4ca-f3db06d36633-kube-api-access-ls64x\") pod \"95aa3a5c-4534-4ca7-a4ca-f3db06d36633\" (UID: \"95aa3a5c-4534-4ca7-a4ca-f3db06d36633\") "
Oct 30 22:10:15.248703 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:15.248580    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/95aa3a5c-4534-4ca7-a4ca-f3db06d36633-metrics-certs\" (UniqueName: \"kubernetes.io/secret/95aa3a5c-4534-4ca7-a4ca-f3db06d36633-metrics-certs\") pod \"95aa3a5c-4534-4ca7-a4ca-f3db06d36633\" (UID: \"95aa3a5c-4534-4ca7-a4ca-f3db06d36633\") "
Oct 30 22:10:15.248768 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:15.248757    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/95aa3a5c-4534-4ca7-a4ca-f3db06d36633-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95aa3a5c-4534-4ca7-a4ca-f3db06d36633-openshift-service-ca\") pod \"95aa3a5c-4534-4ca7-a4ca-f3db06d36633\" (UID: \"95aa3a5c-4534-4ca7-a4ca-f3db06d36633\") "
Oct 30 22:10:15.248861 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:15.248845    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/95aa3a5c-4534-4ca7-a4ca-f3db06d36633-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95aa3a5c-4534-4ca7-a4ca-f3db06d36633-maas-api-tls\") pod \"95aa3a5c-4534-4ca7-a4ca-f3db06d36633\" (UID: \"95aa3a5c-4534-4ca7-a4ca-f3db06d36633\") "
Oct 30 22:10:15.251590 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:15.251388    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95aa3a5c-4534-4ca7-a4ca-f3db06d36633-openshift-service-ca" pod "95aa3a5c-4534-4ca7-a4ca-f3db06d36633" (UID: "95aa3a5c-4534-4ca7-a4ca-f3db06d36633"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:15.253352 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:15.252012    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95aa3a5c-4534-4ca7-a4ca-f3db06d36633-maas-api-tls" pod "95aa3a5c-4534-4ca7-a4ca-f3db06d36633" (UID: "95aa3a5c-4534-4ca7-a4ca-f3db06d36633"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:10:15.253900 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:15.253818    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95aa3a5c-4534-4ca7-a4ca-f3db06d36633-kube-api-access-ls64x" pod "95aa3a5c-4534-4ca7-a4ca-f3db06d36633" (UID: "95aa3a5c-4534-4ca7-a4ca-f3db06d36633"). InnerVolumeSpecName "kube-api-access-ls64x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:15.254570 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:15.254366    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95aa3a5c-4534-4ca7-a4ca-f3db06d36633-metrics-certs" pod "95aa3a5c-4534-4ca7-a4ca-f3db06d36633" (UID: "95aa3a5c-4534-4ca7-a4ca-f3db06d36633"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:10:15.263028 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:15.262998    4871 generic.go:356] "Generic (PLEG): container finished" podID="95aa3a5c-4534-4ca7-a4ca-f3db06d36633" containerID="496f0bb133b41c2c10cbdebd5fa4f5a97c12970bd8d369ffb638e13feed1858e" exitCode=0
Oct 30 22:10:15.263155 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:15.263051    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-97f097-85dddff48-dj9xs"
Oct 30 22:10:15.263155 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:15.263079    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-97f097-85dddff48-dj9xs" event={"ID":"95aa3a5c-4534-4ca7-a4ca-f3db06d36633","Type":"ContainerDied","Data":"496f0bb133b41c2c10cbdebd5fa4f5a97c12970bd8d369ffb638e13feed1858e"}
Oct 30 22:10:15.263155 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:15.263142    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-97f097-85dddff48-dj9xs" event={"ID":"95aa3a5c-4534-4ca7-a4ca-f3db06d36633","Type":"ContainerDied","Data":"9305f23ba13bfd0ff9c51dc0ec59b1ad4ecb99b98bd7c96c66861944cd5fddbe"}
Oct 30 22:10:15.263374 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:15.263172    4871 scope.go:122] "RemoveContainer" containerID="496f0bb133b41c2c10cbdebd5fa4f5a97c12970bd8d369ffb638e13feed1858e"
Oct 30 22:10:15.285985 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:15.285958    4871 scope.go:122] "RemoveContainer" containerID="496f0bb133b41c2c10cbdebd5fa4f5a97c12970bd8d369ffb638e13feed1858e"
Oct 30 22:10:15.286304 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:10:15.286280    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"496f0bb133b41c2c10cbdebd5fa4f5a97c12970bd8d369ffb638e13feed1858e\": container with ID starting with 496f0bb133b41c2c10cbdebd5fa4f5a97c12970bd8d369ffb638e13feed1858e not found: ID does not exist" containerID="496f0bb133b41c2c10cbdebd5fa4f5a97c12970bd8d369ffb638e13feed1858e"
Oct 30 22:10:15.286375 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:15.286312    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"496f0bb133b41c2c10cbdebd5fa4f5a97c12970bd8d369ffb638e13feed1858e"} err="failed to get container status \"496f0bb133b41c2c10cbdebd5fa4f5a97c12970bd8d369ffb638e13feed1858e\": rpc error: code = NotFound desc = could not find container \"496f0bb133b41c2c10cbdebd5fa4f5a97c12970bd8d369ffb638e13feed1858e\": container with ID starting with 496f0bb133b41c2c10cbdebd5fa4f5a97c12970bd8d369ffb638e13feed1858e not found: ID does not exist"
Oct 30 22:10:15.350797 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:15.350706    4871 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95aa3a5c-4534-4ca7-a4ca-f3db06d36633-openshift-service-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:10:15.350797 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:15.350739    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95aa3a5c-4534-4ca7-a4ca-f3db06d36633-maas-api-tls\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:10:15.350797 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:15.350755    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ls64x\" (UniqueName: \"kubernetes.io/projected/95aa3a5c-4534-4ca7-a4ca-f3db06d36633-kube-api-access-ls64x\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:10:15.350797 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:15.350767    4871 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95aa3a5c-4534-4ca7-a4ca-f3db06d36633-metrics-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:10:15.420322 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:15.420289    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-97f097-85dddff48-dj9xs"]
Oct 30 22:10:15.423837 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:15.423804    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-97f097-85dddff48-dj9xs"]
Oct 30 22:10:15.687733 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:15.687694    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:15.854295 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:15.848683    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:15.962607 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:15.962532    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="95aa3a5c-4534-4ca7-a4ca-f3db06d36633" path="/var/lib/kubelet/pods/95aa3a5c-4534-4ca7-a4ca-f3db06d36633/volumes"
Oct 30 22:10:16.008536 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:16.008471    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:16.444429 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:16.444392    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:16.720622 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:16.720522    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:16.878359 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:16.878323    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:17.472399 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:17.472362    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:20.992253 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:20.992200    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:21.266584 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:21.247922    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:21.461334 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:21.458197    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f0b8c9-79d9577847-4wpgv"]
Oct 30 22:10:21.461334 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:21.459213    4871 cpu_manager.go:414] "containerMap: removing container" podUID="95aa3a5c-4534-4ca7-a4ca-f3db06d36633" containerName="maas-api"
Oct 30 22:10:21.461334 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:21.459256    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="95aa3a5c-4534-4ca7-a4ca-f3db06d36633" containerName="maas-api"
Oct 30 22:10:21.461334 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:21.459428    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="95aa3a5c-4534-4ca7-a4ca-f3db06d36633" containerName="maas-api"
Oct 30 22:10:21.468504 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:21.468470    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f0b8c9-79d9577847-4wpgv"
Oct 30 22:10:21.488082 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:21.487817    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f0b8c9-79d9577847-4wpgv"]
Oct 30 22:10:21.545313 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:21.542182    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnswd\" (UniqueName: \"kubernetes.io/projected/661be396-da87-40b1-bfab-7736db05daf0-kube-api-access-dnswd\") pod \"payload-pre-processing-e2e-ipp-a-f0b8c9-79d9577847-4wpgv\" (UID: \"661be396-da87-40b1-bfab-7736db05daf0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f0b8c9-79d9577847-4wpgv"
Oct 30 22:10:21.545313 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:21.542366    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/661be396-da87-40b1-bfab-7736db05daf0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-f0b8c9-79d9577847-4wpgv\" (UID: \"661be396-da87-40b1-bfab-7736db05daf0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f0b8c9-79d9577847-4wpgv"
Oct 30 22:10:21.545313 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:21.543289    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-f0b8c9-dockercfg-2fws7\""
Oct 30 22:10:21.545313 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:21.543378    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-f0b8c9\""
Oct 30 22:10:21.662272 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:21.649421    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/661be396-da87-40b1-bfab-7736db05daf0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-f0b8c9-79d9577847-4wpgv\" (UID: \"661be396-da87-40b1-bfab-7736db05daf0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f0b8c9-79d9577847-4wpgv"
Oct 30 22:10:21.662272 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:21.649625    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnswd\" (UniqueName: \"kubernetes.io/projected/661be396-da87-40b1-bfab-7736db05daf0-kube-api-access-dnswd\") pod \"payload-pre-processing-e2e-ipp-a-f0b8c9-79d9577847-4wpgv\" (UID: \"661be396-da87-40b1-bfab-7736db05daf0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f0b8c9-79d9577847-4wpgv"
Oct 30 22:10:21.662272 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:21.650993    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/661be396-da87-40b1-bfab-7736db05daf0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-f0b8c9-79d9577847-4wpgv\" (UID: \"661be396-da87-40b1-bfab-7736db05daf0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f0b8c9-79d9577847-4wpgv"
Oct 30 22:10:21.770700 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:21.770366    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnswd\" (UniqueName: \"kubernetes.io/projected/661be396-da87-40b1-bfab-7736db05daf0-kube-api-access-dnswd\") pod \"payload-pre-processing-e2e-ipp-a-f0b8c9-79d9577847-4wpgv\" (UID: \"661be396-da87-40b1-bfab-7736db05daf0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f0b8c9-79d9577847-4wpgv"
Oct 30 22:10:21.794408 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:21.794298    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f0b8c9-79d9577847-4wpgv"
Oct 30 22:10:22.233263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:22.232744    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:22.295975 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:22.295905    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f0b8c9-79d9577847-4wpgv"]
Oct 30 22:10:22.299545 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:10:22.299510    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod661be396_da87_40b1_bfab_7736db05daf0.slice/crio-798b60ce5edf4313338fe44d6f0e79dd0ad93a80df99bedba84d932ce30e0c0f WatchSource:0}: Error finding container 798b60ce5edf4313338fe44d6f0e79dd0ad93a80df99bedba84d932ce30e0c0f: Status 404 returned error can't find the container with id 798b60ce5edf4313338fe44d6f0e79dd0ad93a80df99bedba84d932ce30e0c0f
Oct 30 22:10:22.441455 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:22.441416    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f0b8c9-79d9577847-4wpgv" event={"ID":"661be396-da87-40b1-bfab-7736db05daf0","Type":"ContainerStarted","Data":"798b60ce5edf4313338fe44d6f0e79dd0ad93a80df99bedba84d932ce30e0c0f"}
Oct 30 22:10:23.450948 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:23.450908    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f0b8c9-79d9577847-4wpgv" event={"ID":"661be396-da87-40b1-bfab-7736db05daf0","Type":"ContainerStarted","Data":"d8d11a3a1ed8c2ab24567cce37ca565746b1c73a0929cacc40e8e16e5aa8f9ca"}
Oct 30 22:10:23.451453 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:23.451008    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f0b8c9-79d9577847-4wpgv"
Oct 30 22:10:23.568307 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:23.568220    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f0b8c9-79d9577847-4wpgv" podStartSLOduration=2.5677891859999997 podStartE2EDuration="2.567789186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:10:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:10:23.563932475 +0000 UTC m=+1965.741421094" watchObservedRunningTime="2026-10-30 22:10:23.567789186 +0000 UTC m=+1965.745277802"
Oct 30 22:10:24.422261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:24.410753    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:24.567022 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:24.566792    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:24.751346 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:24.749995    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"]
Oct 30 22:10:24.751346 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:24.750281    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm" podUID="d1743672-f86e-4bb4-a827-99344823cc30" containerName="istio-proxy" containerID="cri-o://542057cf992ca8ac9b971afd54855ba1ca9dabef129ae6484adc5a9d46e95c78" gracePeriod=30
Oct 30 22:10:24.788799 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:24.788761    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:25.724256 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:25.724126    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:26.268965 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:26.268928    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"]
Oct 30 22:10:26.273216 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:26.273188    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"
Oct 30 22:10:26.326764 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:26.326728    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-65659d-openshift-default-dockercfg-47dsz\""
Oct 30 22:10:26.390259 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:26.370014    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78387fab-963c-4f00-9fda-33a5a5ebaee7-istio-podinfo\") pod \"e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk\" (UID: \"78387fab-963c-4f00-9fda-33a5a5ebaee7\") " pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"
Oct 30 22:10:26.416654 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:26.416617    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78387fab-963c-4f00-9fda-33a5a5ebaee7-istio-token\") pod \"e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk\" (UID: \"78387fab-963c-4f00-9fda-33a5a5ebaee7\") " pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"
Oct 30 22:10:26.416917 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:26.416897    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78387fab-963c-4f00-9fda-33a5a5ebaee7-workload-certs\") pod \"e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk\" (UID: \"78387fab-963c-4f00-9fda-33a5a5ebaee7\") " pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"
Oct 30 22:10:26.417071 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:26.417056    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78387fab-963c-4f00-9fda-33a5a5ebaee7-istiod-ca-cert\") pod \"e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk\" (UID: \"78387fab-963c-4f00-9fda-33a5a5ebaee7\") " pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"
Oct 30 22:10:26.417289 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:26.417270    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78387fab-963c-4f00-9fda-33a5a5ebaee7-istio-data\") pod \"e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk\" (UID: \"78387fab-963c-4f00-9fda-33a5a5ebaee7\") " pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"
Oct 30 22:10:26.417489 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:26.417462    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkp2m\" (UniqueName: \"kubernetes.io/projected/78387fab-963c-4f00-9fda-33a5a5ebaee7-kube-api-access-zkp2m\") pod \"e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk\" (UID: \"78387fab-963c-4f00-9fda-33a5a5ebaee7\") " pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"
Oct 30 22:10:26.417640 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:26.417625    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78387fab-963c-4f00-9fda-33a5a5ebaee7-credential-socket\") pod \"e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk\" (UID: \"78387fab-963c-4f00-9fda-33a5a5ebaee7\") " pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"
Oct 30 22:10:26.417845 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:26.417831    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78387fab-963c-4f00-9fda-33a5a5ebaee7-workload-socket\") pod \"e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk\" (UID: \"78387fab-963c-4f00-9fda-33a5a5ebaee7\") " pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"
Oct 30 22:10:26.417953 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:26.417940    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78387fab-963c-4f00-9fda-33a5a5ebaee7-istio-envoy\") pod \"e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk\" (UID: \"78387fab-963c-4f00-9fda-33a5a5ebaee7\") " pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"
Oct 30 22:10:26.444308 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:26.425605    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"]
Oct 30 22:10:26.447129 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:26.446880    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:26.520325 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:26.520171    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78387fab-963c-4f00-9fda-33a5a5ebaee7-credential-socket\") pod \"e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk\" (UID: \"78387fab-963c-4f00-9fda-33a5a5ebaee7\") " pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"
Oct 30 22:10:26.520325 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:26.520248    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78387fab-963c-4f00-9fda-33a5a5ebaee7-workload-socket\") pod \"e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk\" (UID: \"78387fab-963c-4f00-9fda-33a5a5ebaee7\") " pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"
Oct 30 22:10:26.520325 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:26.520282    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78387fab-963c-4f00-9fda-33a5a5ebaee7-istio-envoy\") pod \"e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk\" (UID: \"78387fab-963c-4f00-9fda-33a5a5ebaee7\") " pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"
Oct 30 22:10:26.520724 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:26.520392    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78387fab-963c-4f00-9fda-33a5a5ebaee7-istio-podinfo\") pod \"e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk\" (UID: \"78387fab-963c-4f00-9fda-33a5a5ebaee7\") " pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"
Oct 30 22:10:26.520724 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:26.520415    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78387fab-963c-4f00-9fda-33a5a5ebaee7-istio-token\") pod \"e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk\" (UID: \"78387fab-963c-4f00-9fda-33a5a5ebaee7\") " pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"
Oct 30 22:10:26.520724 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:26.520484    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78387fab-963c-4f00-9fda-33a5a5ebaee7-workload-certs\") pod \"e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk\" (UID: \"78387fab-963c-4f00-9fda-33a5a5ebaee7\") " pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"
Oct 30 22:10:26.521106 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:26.521081    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78387fab-963c-4f00-9fda-33a5a5ebaee7-credential-socket\") pod \"e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk\" (UID: \"78387fab-963c-4f00-9fda-33a5a5ebaee7\") " pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"
Oct 30 22:10:26.521202 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:26.521095    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78387fab-963c-4f00-9fda-33a5a5ebaee7-istiod-ca-cert\") pod \"e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk\" (UID: \"78387fab-963c-4f00-9fda-33a5a5ebaee7\") " pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"
Oct 30 22:10:26.521343 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:26.521227    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78387fab-963c-4f00-9fda-33a5a5ebaee7-istio-data\") pod \"e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk\" (UID: \"78387fab-963c-4f00-9fda-33a5a5ebaee7\") " pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"
Oct 30 22:10:26.521446 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:26.521354    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkp2m\" (UniqueName: \"kubernetes.io/projected/78387fab-963c-4f00-9fda-33a5a5ebaee7-kube-api-access-zkp2m\") pod \"e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk\" (UID: \"78387fab-963c-4f00-9fda-33a5a5ebaee7\") " pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"
Oct 30 22:10:26.521807 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:26.521781    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78387fab-963c-4f00-9fda-33a5a5ebaee7-istiod-ca-cert\") pod \"e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk\" (UID: \"78387fab-963c-4f00-9fda-33a5a5ebaee7\") " pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"
Oct 30 22:10:26.521930 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:26.521858    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78387fab-963c-4f00-9fda-33a5a5ebaee7-workload-socket\") pod \"e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk\" (UID: \"78387fab-963c-4f00-9fda-33a5a5ebaee7\") " pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"
Oct 30 22:10:26.522633 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:26.522608    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78387fab-963c-4f00-9fda-33a5a5ebaee7-workload-certs\") pod \"e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk\" (UID: \"78387fab-963c-4f00-9fda-33a5a5ebaee7\") " pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"
Oct 30 22:10:26.522797 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:26.522770    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78387fab-963c-4f00-9fda-33a5a5ebaee7-istio-data\") pod \"e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk\" (UID: \"78387fab-963c-4f00-9fda-33a5a5ebaee7\") " pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"
Oct 30 22:10:26.526679 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:26.526635    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78387fab-963c-4f00-9fda-33a5a5ebaee7-istio-envoy\") pod \"e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk\" (UID: \"78387fab-963c-4f00-9fda-33a5a5ebaee7\") " pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"
Oct 30 22:10:26.531054 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:26.531025    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78387fab-963c-4f00-9fda-33a5a5ebaee7-istio-podinfo\") pod \"e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk\" (UID: \"78387fab-963c-4f00-9fda-33a5a5ebaee7\") " pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"
Oct 30 22:10:26.580292 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:26.580209    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78387fab-963c-4f00-9fda-33a5a5ebaee7-istio-token\") pod \"e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk\" (UID: \"78387fab-963c-4f00-9fda-33a5a5ebaee7\") " pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"
Oct 30 22:10:26.580483 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:26.580373    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkp2m\" (UniqueName: \"kubernetes.io/projected/78387fab-963c-4f00-9fda-33a5a5ebaee7-kube-api-access-zkp2m\") pod \"e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk\" (UID: \"78387fab-963c-4f00-9fda-33a5a5ebaee7\") " pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"
Oct 30 22:10:26.590737 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:26.590702    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"
Oct 30 22:10:26.888279 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:26.874641    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:27.043382 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:27.043344    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"]
Oct 30 22:10:27.043811 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:10:27.043776    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78387fab_963c_4f00_9fda_33a5a5ebaee7.slice/crio-f2bbca7b2ba5bebb83f7d1ed044f4b494a30fa84c0584d55d1f7686f9ffbe30e WatchSource:0}: Error finding container f2bbca7b2ba5bebb83f7d1ed044f4b494a30fa84c0584d55d1f7686f9ffbe30e: Status 404 returned error can't find the container with id f2bbca7b2ba5bebb83f7d1ed044f4b494a30fa84c0584d55d1f7686f9ffbe30e
Oct 30 22:10:27.046666 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:27.046627    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:10:27.046789 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:27.046711    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:10:27.046789 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:27.046760    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:10:27.256603 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:27.256489    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:27.497091 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:27.497045    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk" event={"ID":"78387fab-963c-4f00-9fda-33a5a5ebaee7","Type":"ContainerStarted","Data":"dd37dcafa918dc0f028207aebe5fd1fd38fca109ef8ab1c431aa6ca3b78ed693"}
Oct 30 22:10:27.497316 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:27.497115    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk" event={"ID":"78387fab-963c-4f00-9fda-33a5a5ebaee7","Type":"ContainerStarted","Data":"f2bbca7b2ba5bebb83f7d1ed044f4b494a30fa84c0584d55d1f7686f9ffbe30e"}
Oct 30 22:10:27.591720 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:27.591633    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"
Oct 30 22:10:27.617494 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:27.617406    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk" podStartSLOduration=1.617377989 podStartE2EDuration="1.617377989s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:10:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:10:27.611964957 +0000 UTC m=+1969.789453637" watchObservedRunningTime="2026-10-30 22:10:27.617377989 +0000 UTC m=+1969.794866606"
Oct 30 22:10:28.598911 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:28.598877    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"
Oct 30 22:10:29.553396 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:29.539342    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"
Oct 30 22:10:29.554661 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:29.554627    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"
Oct 30 22:10:29.992966 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:29.992934    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"
Oct 30 22:10:30.117447 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.117401    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1743672-f86e-4bb4-a827-99344823cc30-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1743672-f86e-4bb4-a827-99344823cc30-workload-socket\") pod \"d1743672-f86e-4bb4-a827-99344823cc30\" (UID: \"d1743672-f86e-4bb4-a827-99344823cc30\") "
Oct 30 22:10:30.117673 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.117535    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d1743672-f86e-4bb4-a827-99344823cc30-istio-token\" (UniqueName: \"kubernetes.io/projected/d1743672-f86e-4bb4-a827-99344823cc30-istio-token\") pod \"d1743672-f86e-4bb4-a827-99344823cc30\" (UID: \"d1743672-f86e-4bb4-a827-99344823cc30\") "
Oct 30 22:10:30.117673 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.117603    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d1743672-f86e-4bb4-a827-99344823cc30-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1743672-f86e-4bb4-a827-99344823cc30-istio-podinfo\") pod \"d1743672-f86e-4bb4-a827-99344823cc30\" (UID: \"d1743672-f86e-4bb4-a827-99344823cc30\") "
Oct 30 22:10:30.117673 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.117652    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1743672-f86e-4bb4-a827-99344823cc30-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1743672-f86e-4bb4-a827-99344823cc30-workload-certs\") pod \"d1743672-f86e-4bb4-a827-99344823cc30\" (UID: \"d1743672-f86e-4bb4-a827-99344823cc30\") "
Oct 30 22:10:30.117860 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.117705    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d1743672-f86e-4bb4-a827-99344823cc30-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1743672-f86e-4bb4-a827-99344823cc30-istiod-ca-cert\") pod \"d1743672-f86e-4bb4-a827-99344823cc30\" (UID: \"d1743672-f86e-4bb4-a827-99344823cc30\") "
Oct 30 22:10:30.117860 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.117813    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d1743672-f86e-4bb4-a827-99344823cc30-kube-api-access-9f86p\" (UniqueName: \"kubernetes.io/projected/d1743672-f86e-4bb4-a827-99344823cc30-kube-api-access-9f86p\") pod \"d1743672-f86e-4bb4-a827-99344823cc30\" (UID: \"d1743672-f86e-4bb4-a827-99344823cc30\") "
Oct 30 22:10:30.117860 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.117856    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1743672-f86e-4bb4-a827-99344823cc30-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1743672-f86e-4bb4-a827-99344823cc30-istio-envoy\") pod \"d1743672-f86e-4bb4-a827-99344823cc30\" (UID: \"d1743672-f86e-4bb4-a827-99344823cc30\") "
Oct 30 22:10:30.119423 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.119355    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1743672-f86e-4bb4-a827-99344823cc30-workload-certs" pod "d1743672-f86e-4bb4-a827-99344823cc30" (UID: "d1743672-f86e-4bb4-a827-99344823cc30"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:30.119423 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.119371    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1743672-f86e-4bb4-a827-99344823cc30-workload-socket" pod "d1743672-f86e-4bb4-a827-99344823cc30" (UID: "d1743672-f86e-4bb4-a827-99344823cc30"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:30.119676 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.117909    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1743672-f86e-4bb4-a827-99344823cc30-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1743672-f86e-4bb4-a827-99344823cc30-credential-socket\") pod \"d1743672-f86e-4bb4-a827-99344823cc30\" (UID: \"d1743672-f86e-4bb4-a827-99344823cc30\") "
Oct 30 22:10:30.119676 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.119655    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1743672-f86e-4bb4-a827-99344823cc30-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1743672-f86e-4bb4-a827-99344823cc30-istio-data\") pod \"d1743672-f86e-4bb4-a827-99344823cc30\" (UID: \"d1743672-f86e-4bb4-a827-99344823cc30\") "
Oct 30 22:10:30.126556 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.119922    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1743672-f86e-4bb4-a827-99344823cc30-istiod-ca-cert" pod "d1743672-f86e-4bb4-a827-99344823cc30" (UID: "d1743672-f86e-4bb4-a827-99344823cc30"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:30.126556 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.120137    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1743672-f86e-4bb4-a827-99344823cc30-istio-data" pod "d1743672-f86e-4bb4-a827-99344823cc30" (UID: "d1743672-f86e-4bb4-a827-99344823cc30"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:30.126556 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.120198    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1743672-f86e-4bb4-a827-99344823cc30-credential-socket" pod "d1743672-f86e-4bb4-a827-99344823cc30" (UID: "d1743672-f86e-4bb4-a827-99344823cc30"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:30.126556 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.120340    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1743672-f86e-4bb4-a827-99344823cc30-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:10:30.126556 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.120358    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1743672-f86e-4bb4-a827-99344823cc30-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:10:30.126556 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.120372    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1743672-f86e-4bb4-a827-99344823cc30-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:10:30.126556 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.120388    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1743672-f86e-4bb4-a827-99344823cc30-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:10:30.126556 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.120406    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1743672-f86e-4bb4-a827-99344823cc30-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:10:30.126556 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.123068    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1743672-f86e-4bb4-a827-99344823cc30-kube-api-access-9f86p" pod "d1743672-f86e-4bb4-a827-99344823cc30" (UID: "d1743672-f86e-4bb4-a827-99344823cc30"). InnerVolumeSpecName "kube-api-access-9f86p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:30.126556 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.124465    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1743672-f86e-4bb4-a827-99344823cc30-istio-token" pod "d1743672-f86e-4bb4-a827-99344823cc30" (UID: "d1743672-f86e-4bb4-a827-99344823cc30"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:30.126556 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.124568    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1743672-f86e-4bb4-a827-99344823cc30-istio-envoy" pod "d1743672-f86e-4bb4-a827-99344823cc30" (UID: "d1743672-f86e-4bb4-a827-99344823cc30"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:30.127325 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.126923    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d1743672-f86e-4bb4-a827-99344823cc30-istio-podinfo" pod "d1743672-f86e-4bb4-a827-99344823cc30" (UID: "d1743672-f86e-4bb4-a827-99344823cc30"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:10:30.221738 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.221689    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9f86p\" (UniqueName: \"kubernetes.io/projected/d1743672-f86e-4bb4-a827-99344823cc30-kube-api-access-9f86p\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:10:30.221738 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.221740    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1743672-f86e-4bb4-a827-99344823cc30-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:10:30.222025 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.221755    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1743672-f86e-4bb4-a827-99344823cc30-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:10:30.222025 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.221768    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1743672-f86e-4bb4-a827-99344823cc30-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:10:30.292934 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.292901    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:30.473360 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.473193    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:30.545000 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.544958    4871 generic.go:356] "Generic (PLEG): container finished" podID="d1743672-f86e-4bb4-a827-99344823cc30" containerID="542057cf992ca8ac9b971afd54855ba1ca9dabef129ae6484adc5a9d46e95c78" exitCode=0
Oct 30 22:10:30.545203 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.545044    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"
Oct 30 22:10:30.545203 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.545055    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm" event={"ID":"d1743672-f86e-4bb4-a827-99344823cc30","Type":"ContainerDied","Data":"542057cf992ca8ac9b971afd54855ba1ca9dabef129ae6484adc5a9d46e95c78"}
Oct 30 22:10:30.545203 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.545144    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm" event={"ID":"d1743672-f86e-4bb4-a827-99344823cc30","Type":"ContainerDied","Data":"2dc766b24fa4e8b8189013aacda6afb8ef229099a8269b600d2573d400dc8c19"}
Oct 30 22:10:30.545203 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.545176    4871 scope.go:122] "RemoveContainer" containerID="542057cf992ca8ac9b971afd54855ba1ca9dabef129ae6484adc5a9d46e95c78"
Oct 30 22:10:30.547561 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.547536    4871 generic.go:356] "Generic (PLEG): container finished" podID="088f4439-453d-406a-b54a-837e10b36efd" containerID="df09c5f2757e1fed96666c817850397aac172e4d3efb7e932f8e6439ae47cc98" exitCode=0
Oct 30 22:10:30.547693 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.547620    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh" event={"ID":"088f4439-453d-406a-b54a-837e10b36efd","Type":"ContainerDied","Data":"df09c5f2757e1fed96666c817850397aac172e4d3efb7e932f8e6439ae47cc98"}
Oct 30 22:10:30.548348 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.548331    4871 scope.go:122] "RemoveContainer" containerID="df09c5f2757e1fed96666c817850397aac172e4d3efb7e932f8e6439ae47cc98"
Oct 30 22:10:30.550049 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.550009    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:10:30.550137 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.550110    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:10:30.550217 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.550158    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:10:30.558575 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.558554    4871 scope.go:122] "RemoveContainer" containerID="4363fcea3e6f82ba8fc2ddac054384b0cc3df2821c66111ae176c14635aaea1c"
Oct 30 22:10:30.569177 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.569147    4871 scope.go:122] "RemoveContainer" containerID="542057cf992ca8ac9b971afd54855ba1ca9dabef129ae6484adc5a9d46e95c78"
Oct 30 22:10:30.569650 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:10:30.569566    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"542057cf992ca8ac9b971afd54855ba1ca9dabef129ae6484adc5a9d46e95c78\": container with ID starting with 542057cf992ca8ac9b971afd54855ba1ca9dabef129ae6484adc5a9d46e95c78 not found: ID does not exist" containerID="542057cf992ca8ac9b971afd54855ba1ca9dabef129ae6484adc5a9d46e95c78"
Oct 30 22:10:30.569650 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.569599    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"542057cf992ca8ac9b971afd54855ba1ca9dabef129ae6484adc5a9d46e95c78"} err="failed to get container status \"542057cf992ca8ac9b971afd54855ba1ca9dabef129ae6484adc5a9d46e95c78\": rpc error: code = NotFound desc = could not find container \"542057cf992ca8ac9b971afd54855ba1ca9dabef129ae6484adc5a9d46e95c78\": container with ID starting with 542057cf992ca8ac9b971afd54855ba1ca9dabef129ae6484adc5a9d46e95c78 not found: ID does not exist"
Oct 30 22:10:30.569650 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.569618    4871 scope.go:122] "RemoveContainer" containerID="4363fcea3e6f82ba8fc2ddac054384b0cc3df2821c66111ae176c14635aaea1c"
Oct 30 22:10:30.569942 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:10:30.569921    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4363fcea3e6f82ba8fc2ddac054384b0cc3df2821c66111ae176c14635aaea1c\": container with ID starting with 4363fcea3e6f82ba8fc2ddac054384b0cc3df2821c66111ae176c14635aaea1c not found: ID does not exist" containerID="4363fcea3e6f82ba8fc2ddac054384b0cc3df2821c66111ae176c14635aaea1c"
Oct 30 22:10:30.570001 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.569952    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4363fcea3e6f82ba8fc2ddac054384b0cc3df2821c66111ae176c14635aaea1c"} err="failed to get container status \"4363fcea3e6f82ba8fc2ddac054384b0cc3df2821c66111ae176c14635aaea1c\": rpc error: code = NotFound desc = could not find container \"4363fcea3e6f82ba8fc2ddac054384b0cc3df2821c66111ae176c14635aaea1c\": container with ID starting with 4363fcea3e6f82ba8fc2ddac054384b0cc3df2821c66111ae176c14635aaea1c not found: ID does not exist"
Oct 30 22:10:30.618746 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.618717    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"
Oct 30 22:10:30.628618 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.628579    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:30.844422 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.844319    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"]
Oct 30 22:10:30.853137 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:30.853090    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-97f097-openshift-default-589b54574c-469zm"]
Oct 30 22:10:31.580625 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:31.580589    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh" event={"ID":"088f4439-453d-406a-b54a-837e10b36efd","Type":"ContainerStarted","Data":"6d11c99737badb09049032029ea5a0df9f82d701d2535c86fccbcd4089bb88ed"}
Oct 30 22:10:31.892353 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:31.892314    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"
Oct 30 22:10:31.893798 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:31.893765    4871 patch_prober.go:29] interesting pod/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused" start-of-body=
Oct 30 22:10:31.893933 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:31.893829    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh" podUID="088f4439-453d-406a-b54a-837e10b36efd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused"
Oct 30 22:10:31.960731 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:31.960689    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d1743672-f86e-4bb4-a827-99344823cc30" path="/var/lib/kubelet/pods/d1743672-f86e-4bb4-a827-99344823cc30/volumes"
Oct 30 22:10:32.398685 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:32.398652    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:32.892575 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:32.892539    4871 patch_prober.go:29] interesting pod/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused" start-of-body=
Oct 30 22:10:32.893025 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:32.892608    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh" podUID="088f4439-453d-406a-b54a-837e10b36efd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused"
Oct 30 22:10:33.892588 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:33.892554    4871 patch_prober.go:29] interesting pod/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused" start-of-body=
Oct 30 22:10:33.892982 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:33.892613    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh" podUID="088f4439-453d-406a-b54a-837e10b36efd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused"
Oct 30 22:10:34.461623 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:34.461584    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f0b8c9-79d9577847-4wpgv"
Oct 30 22:10:34.891812 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:34.891778    4871 patch_prober.go:29] interesting pod/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused" start-of-body=
Oct 30 22:10:34.892013 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:34.891833    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh" podUID="088f4439-453d-406a-b54a-837e10b36efd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused"
Oct 30 22:10:34.955691 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:34.955657    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"]
Oct 30 22:10:34.956720 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:34.956698    4871 cpu_manager.go:414] "containerMap: removing container" podUID="d1743672-f86e-4bb4-a827-99344823cc30" containerName="istio-proxy"
Oct 30 22:10:34.956843 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:34.956833    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d1743672-f86e-4bb4-a827-99344823cc30" containerName="istio-proxy"
Oct 30 22:10:34.956932 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:34.956924    4871 cpu_manager.go:414] "containerMap: removing container" podUID="d1743672-f86e-4bb4-a827-99344823cc30" containerName="istio-proxy"
Oct 30 22:10:34.957041 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:34.957032    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d1743672-f86e-4bb4-a827-99344823cc30" containerName="istio-proxy"
Oct 30 22:10:34.957258 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:34.957232    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="d1743672-f86e-4bb4-a827-99344823cc30" containerName="istio-proxy"
Oct 30 22:10:34.962342 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:34.962323    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"
Oct 30 22:10:34.998480 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:34.998444    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"]
Oct 30 22:10:34.999248 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:34.999210    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="d1743672-f86e-4bb4-a827-99344823cc30" containerName="istio-proxy"
Oct 30 22:10:35.030267 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.026411    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"]
Oct 30 22:10:35.030474 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.030381    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"
Oct 30 22:10:35.056505 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.054968    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"
Oct 30 22:10:35.056505 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.055745    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"]
Oct 30 22:10:35.084619 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.084586    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"]
Oct 30 22:10:35.091148 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.091127    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"]
Oct 30 22:10:35.094013 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.093986    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7df854bf67-85wfj\" (UID: \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"
Oct 30 22:10:35.094181 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.094026    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-workload-certs\") pod \"maas-default-gateway-openshift-default-7df854bf67-2cvb2\" (UID: \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"
Oct 30 22:10:35.094181 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.094057    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-credential-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-2cvb2\" (UID: \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"
Oct 30 22:10:35.094181 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.094098    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-istio-envoy\") pod \"maas-default-gateway-openshift-default-7df854bf67-85wfj\" (UID: \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"
Oct 30 22:10:35.094181 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.094129    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wdh5\" (UniqueName: \"kubernetes.io/projected/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-kube-api-access-6wdh5\") pod \"maas-default-gateway-openshift-default-7df854bf67-85wfj\" (UID: \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"
Oct 30 22:10:35.094422 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.094212    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-istio-data\") pod \"maas-default-gateway-openshift-default-7df854bf67-85wfj\" (UID: \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"
Oct 30 22:10:35.094422 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.094297    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-workload-certs\") pod \"maas-default-gateway-openshift-default-7df854bf67-85wfj\" (UID: \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"
Oct 30 22:10:35.094422 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.094380    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7df854bf67-85wfj\" (UID: \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"
Oct 30 22:10:35.094422 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.094406    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-istio-envoy\") pod \"maas-default-gateway-openshift-default-7df854bf67-2cvb2\" (UID: \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"
Oct 30 22:10:35.094632 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.094429    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7df854bf67-2cvb2\" (UID: \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"
Oct 30 22:10:35.094632 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.094459    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-istio-token\") pod \"maas-default-gateway-openshift-default-7df854bf67-2cvb2\" (UID: \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"
Oct 30 22:10:35.094632 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.094493    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-workload-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-2cvb2\" (UID: \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"
Oct 30 22:10:35.094632 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.094548    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7df854bf67-2cvb2\" (UID: \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"
Oct 30 22:10:35.094632 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.094589    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zckr\" (UniqueName: \"kubernetes.io/projected/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-kube-api-access-6zckr\") pod \"maas-default-gateway-openshift-default-7df854bf67-2cvb2\" (UID: \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"
Oct 30 22:10:35.094632 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.094626    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-workload-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-85wfj\" (UID: \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"
Oct 30 22:10:35.094938 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.094661    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-istio-token\") pod \"maas-default-gateway-openshift-default-7df854bf67-85wfj\" (UID: \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"
Oct 30 22:10:35.094938 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.094693    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-credential-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-85wfj\" (UID: \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"
Oct 30 22:10:35.094938 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.094783    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-istio-data\") pod \"maas-default-gateway-openshift-default-7df854bf67-2cvb2\" (UID: \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"
Oct 30 22:10:35.196780 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.196697    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa8f3772-b525-4512-a629-603102f076c0-istio-data\") pod \"maas-default-gateway-openshift-default-7df854bf67-jbljg\" (UID: \"fa8f3772-b525-4512-a629-603102f076c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"
Oct 30 22:10:35.196780 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.196741    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjpfg\" (UniqueName: \"kubernetes.io/projected/fa8f3772-b525-4512-a629-603102f076c0-kube-api-access-fjpfg\") pod \"maas-default-gateway-openshift-default-7df854bf67-jbljg\" (UID: \"fa8f3772-b525-4512-a629-603102f076c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"
Oct 30 22:10:35.196780 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.196768    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7df854bf67-85wfj\" (UID: \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"
Oct 30 22:10:35.197007 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.196954    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-workload-certs\") pod \"maas-default-gateway-openshift-default-7df854bf67-2cvb2\" (UID: \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"
Oct 30 22:10:35.197007 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.196991    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-credential-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-2cvb2\" (UID: \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"
Oct 30 22:10:35.197105 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.197024    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-istio-envoy\") pod \"maas-default-gateway-openshift-default-7df854bf67-85wfj\" (UID: \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"
Oct 30 22:10:35.197105 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.197047    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wdh5\" (UniqueName: \"kubernetes.io/projected/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-kube-api-access-6wdh5\") pod \"maas-default-gateway-openshift-default-7df854bf67-85wfj\" (UID: \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"
Oct 30 22:10:35.197214 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.197112    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-istio-data\") pod \"maas-default-gateway-openshift-default-7df854bf67-85wfj\" (UID: \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"
Oct 30 22:10:35.197214 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.197142    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa8f3772-b525-4512-a629-603102f076c0-workload-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-jbljg\" (UID: \"fa8f3772-b525-4512-a629-603102f076c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"
Oct 30 22:10:35.197336 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.197276    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-workload-certs\") pod \"maas-default-gateway-openshift-default-7df854bf67-85wfj\" (UID: \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"
Oct 30 22:10:35.197395 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.197338    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa8f3772-b525-4512-a629-603102f076c0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7df854bf67-jbljg\" (UID: \"fa8f3772-b525-4512-a629-603102f076c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"
Oct 30 22:10:35.197395 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.197369    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa8f3772-b525-4512-a629-603102f076c0-istio-envoy\") pod \"maas-default-gateway-openshift-default-7df854bf67-jbljg\" (UID: \"fa8f3772-b525-4512-a629-603102f076c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"
Oct 30 22:10:35.197513 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.197405    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa8f3772-b525-4512-a629-603102f076c0-workload-certs\") pod \"maas-default-gateway-openshift-default-7df854bf67-jbljg\" (UID: \"fa8f3772-b525-4512-a629-603102f076c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"
Oct 30 22:10:35.197513 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.197408    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-workload-certs\") pod \"maas-default-gateway-openshift-default-7df854bf67-2cvb2\" (UID: \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"
Oct 30 22:10:35.197513 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.197428    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7df854bf67-85wfj\" (UID: \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"
Oct 30 22:10:35.197513 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.197458    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-credential-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-2cvb2\" (UID: \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"
Oct 30 22:10:35.197513 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.197465    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-istio-envoy\") pod \"maas-default-gateway-openshift-default-7df854bf67-2cvb2\" (UID: \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"
Oct 30 22:10:35.197513 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.197471    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-istio-data\") pod \"maas-default-gateway-openshift-default-7df854bf67-85wfj\" (UID: \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"
Oct 30 22:10:35.197834 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.197580    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7df854bf67-2cvb2\" (UID: \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"
Oct 30 22:10:35.197834 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.197637    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-istio-token\") pod \"maas-default-gateway-openshift-default-7df854bf67-2cvb2\" (UID: \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"
Oct 30 22:10:35.197834 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.197771    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-workload-certs\") pod \"maas-default-gateway-openshift-default-7df854bf67-85wfj\" (UID: \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"
Oct 30 22:10:35.197993 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.197674    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa8f3772-b525-4512-a629-603102f076c0-istio-token\") pod \"maas-default-gateway-openshift-default-7df854bf67-jbljg\" (UID: \"fa8f3772-b525-4512-a629-603102f076c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"
Oct 30 22:10:35.197993 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.197968    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-workload-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-2cvb2\" (UID: \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"
Oct 30 22:10:35.197993 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.197980    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7df854bf67-85wfj\" (UID: \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"
Oct 30 22:10:35.198157 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.197997    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa8f3772-b525-4512-a629-603102f076c0-credential-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-jbljg\" (UID: \"fa8f3772-b525-4512-a629-603102f076c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"
Oct 30 22:10:35.198157 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.198027    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa8f3772-b525-4512-a629-603102f076c0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7df854bf67-jbljg\" (UID: \"fa8f3772-b525-4512-a629-603102f076c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"
Oct 30 22:10:35.198157 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.198064    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7df854bf67-2cvb2\" (UID: \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"
Oct 30 22:10:35.198157 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.198106    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zckr\" (UniqueName: \"kubernetes.io/projected/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-kube-api-access-6zckr\") pod \"maas-default-gateway-openshift-default-7df854bf67-2cvb2\" (UID: \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"
Oct 30 22:10:35.198157 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.198139    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-workload-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-85wfj\" (UID: \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"
Oct 30 22:10:35.198421 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.198164    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-istio-token\") pod \"maas-default-gateway-openshift-default-7df854bf67-85wfj\" (UID: \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"
Oct 30 22:10:35.198421 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.198194    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-credential-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-85wfj\" (UID: \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"
Oct 30 22:10:35.198421 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.198255    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-workload-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-2cvb2\" (UID: \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"
Oct 30 22:10:35.198421 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.198288    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-istio-data\") pod \"maas-default-gateway-openshift-default-7df854bf67-2cvb2\" (UID: \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"
Oct 30 22:10:35.198730 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.198708    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-istio-data\") pod \"maas-default-gateway-openshift-default-7df854bf67-2cvb2\" (UID: \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"
Oct 30 22:10:35.198945 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.198921    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-workload-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-85wfj\" (UID: \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"
Oct 30 22:10:35.199256 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.199223    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-credential-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-85wfj\" (UID: \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"
Oct 30 22:10:35.199375 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.199273    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7df854bf67-2cvb2\" (UID: \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"
Oct 30 22:10:35.199959 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.199935    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7df854bf67-85wfj\" (UID: \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"
Oct 30 22:10:35.200054 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.200017    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-istio-envoy\") pod \"maas-default-gateway-openshift-default-7df854bf67-2cvb2\" (UID: \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"
Oct 30 22:10:35.200356 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.200337    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7df854bf67-2cvb2\" (UID: \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"
Oct 30 22:10:35.200434 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.200414    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-istio-envoy\") pod \"maas-default-gateway-openshift-default-7df854bf67-85wfj\" (UID: \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"
Oct 30 22:10:35.256333 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.256257    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-istio-token\") pod \"maas-default-gateway-openshift-default-7df854bf67-2cvb2\" (UID: \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"
Oct 30 22:10:35.256333 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.256276    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wdh5\" (UniqueName: \"kubernetes.io/projected/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-kube-api-access-6wdh5\") pod \"maas-default-gateway-openshift-default-7df854bf67-85wfj\" (UID: \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"
Oct 30 22:10:35.257300 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.257273    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-istio-token\") pod \"maas-default-gateway-openshift-default-7df854bf67-85wfj\" (UID: \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"
Oct 30 22:10:35.257504 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.257473    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zckr\" (UniqueName: \"kubernetes.io/projected/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-kube-api-access-6zckr\") pod \"maas-default-gateway-openshift-default-7df854bf67-2cvb2\" (UID: \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"
Oct 30 22:10:35.280524 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.280488    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"
Oct 30 22:10:35.300152 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.300121    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa8f3772-b525-4512-a629-603102f076c0-workload-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-jbljg\" (UID: \"fa8f3772-b525-4512-a629-603102f076c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"
Oct 30 22:10:35.300269 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.300188    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa8f3772-b525-4512-a629-603102f076c0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7df854bf67-jbljg\" (UID: \"fa8f3772-b525-4512-a629-603102f076c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"
Oct 30 22:10:35.300269 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.300219    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa8f3772-b525-4512-a629-603102f076c0-istio-envoy\") pod \"maas-default-gateway-openshift-default-7df854bf67-jbljg\" (UID: \"fa8f3772-b525-4512-a629-603102f076c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"
Oct 30 22:10:35.300350 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.300290    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa8f3772-b525-4512-a629-603102f076c0-workload-certs\") pod \"maas-default-gateway-openshift-default-7df854bf67-jbljg\" (UID: \"fa8f3772-b525-4512-a629-603102f076c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"
Oct 30 22:10:35.300350 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.300331    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa8f3772-b525-4512-a629-603102f076c0-istio-token\") pod \"maas-default-gateway-openshift-default-7df854bf67-jbljg\" (UID: \"fa8f3772-b525-4512-a629-603102f076c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"
Oct 30 22:10:35.300425 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.300362    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa8f3772-b525-4512-a629-603102f076c0-credential-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-jbljg\" (UID: \"fa8f3772-b525-4512-a629-603102f076c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"
Oct 30 22:10:35.300425 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.300390    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa8f3772-b525-4512-a629-603102f076c0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7df854bf67-jbljg\" (UID: \"fa8f3772-b525-4512-a629-603102f076c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"
Oct 30 22:10:35.300602 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.300577    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa8f3772-b525-4512-a629-603102f076c0-workload-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-jbljg\" (UID: \"fa8f3772-b525-4512-a629-603102f076c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"
Oct 30 22:10:35.300642 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.300629    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa8f3772-b525-4512-a629-603102f076c0-istio-data\") pod \"maas-default-gateway-openshift-default-7df854bf67-jbljg\" (UID: \"fa8f3772-b525-4512-a629-603102f076c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"
Oct 30 22:10:35.300684 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.300661    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjpfg\" (UniqueName: \"kubernetes.io/projected/fa8f3772-b525-4512-a629-603102f076c0-kube-api-access-fjpfg\") pod \"maas-default-gateway-openshift-default-7df854bf67-jbljg\" (UID: \"fa8f3772-b525-4512-a629-603102f076c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"
Oct 30 22:10:35.301439 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.300835    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa8f3772-b525-4512-a629-603102f076c0-workload-certs\") pod \"maas-default-gateway-openshift-default-7df854bf67-jbljg\" (UID: \"fa8f3772-b525-4512-a629-603102f076c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"
Oct 30 22:10:35.301439 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.300969    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa8f3772-b525-4512-a629-603102f076c0-credential-socket\") pod \"maas-default-gateway-openshift-default-7df854bf67-jbljg\" (UID: \"fa8f3772-b525-4512-a629-603102f076c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"
Oct 30 22:10:35.301575 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.301491    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa8f3772-b525-4512-a629-603102f076c0-istio-data\") pod \"maas-default-gateway-openshift-default-7df854bf67-jbljg\" (UID: \"fa8f3772-b525-4512-a629-603102f076c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"
Oct 30 22:10:35.301575 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.301526    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa8f3772-b525-4512-a629-603102f076c0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7df854bf67-jbljg\" (UID: \"fa8f3772-b525-4512-a629-603102f076c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"
Oct 30 22:10:35.338264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.336891    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa8f3772-b525-4512-a629-603102f076c0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7df854bf67-jbljg\" (UID: \"fa8f3772-b525-4512-a629-603102f076c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"
Oct 30 22:10:35.339025 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.338943    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa8f3772-b525-4512-a629-603102f076c0-istio-envoy\") pod \"maas-default-gateway-openshift-default-7df854bf67-jbljg\" (UID: \"fa8f3772-b525-4512-a629-603102f076c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"
Oct 30 22:10:35.360098 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.359766    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa8f3772-b525-4512-a629-603102f076c0-istio-token\") pod \"maas-default-gateway-openshift-default-7df854bf67-jbljg\" (UID: \"fa8f3772-b525-4512-a629-603102f076c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"
Oct 30 22:10:35.360098 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.360046    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjpfg\" (UniqueName: \"kubernetes.io/projected/fa8f3772-b525-4512-a629-603102f076c0-kube-api-access-fjpfg\") pod \"maas-default-gateway-openshift-default-7df854bf67-jbljg\" (UID: \"fa8f3772-b525-4512-a629-603102f076c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"
Oct 30 22:10:35.378766 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.378705    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"
Oct 30 22:10:35.394163 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.394133    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"
Oct 30 22:10:35.590444 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:10:35.590398    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:10:35.752387 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.752350    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"]
Oct 30 22:10:35.754328 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:10:35.754296    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd6f1baa_ab90_488d_abe5_e70b4f6298ce.slice/crio-014bfbc48312a70ef299b38bf05ade33d17794c6bd4217276d9b88bd1da2ea0f WatchSource:0}: Error finding container 014bfbc48312a70ef299b38bf05ade33d17794c6bd4217276d9b88bd1da2ea0f: Status 404 returned error can't find the container with id 014bfbc48312a70ef299b38bf05ade33d17794c6bd4217276d9b88bd1da2ea0f
Oct 30 22:10:35.757691 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.757656    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:10:35.757787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.757725    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:10:35.757787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.757755    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:10:35.841713 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:10:35.841675    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f13966e_b7ac_4a97_89b0_f98a9b44c5ec.slice/crio-9e5e93eb3b38d5e421bed4cbd85656bd95d8bbbe7ccc4e19e600996e9bfca7a3 WatchSource:0}: Error finding container 9e5e93eb3b38d5e421bed4cbd85656bd95d8bbbe7ccc4e19e600996e9bfca7a3: Status 404 returned error can't find the container with id 9e5e93eb3b38d5e421bed4cbd85656bd95d8bbbe7ccc4e19e600996e9bfca7a3
Oct 30 22:10:35.842577 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.842553    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"]
Oct 30 22:10:35.844394 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.844361    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:10:35.844490 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.844428    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:10:35.844490 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.844464    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:10:35.874582 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.874550    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"]
Oct 30 22:10:35.877280 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:10:35.876589    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa8f3772_b525_4512_a629_603102f076c0.slice/crio-ddad1ddd23d08b26ebfd6fbca87f14bab367732c94cdcfc448ff95378a7d7ae6 WatchSource:0}: Error finding container ddad1ddd23d08b26ebfd6fbca87f14bab367732c94cdcfc448ff95378a7d7ae6: Status 404 returned error can't find the container with id ddad1ddd23d08b26ebfd6fbca87f14bab367732c94cdcfc448ff95378a7d7ae6
Oct 30 22:10:35.879975 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.879937    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:10:35.880054 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.880036    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:10:35.880100 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.880083    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:10:35.893013 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.892984    4871 patch_prober.go:29] interesting pod/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused" start-of-body=
Oct 30 22:10:35.893140 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:35.893055    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh" podUID="088f4439-453d-406a-b54a-837e10b36efd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused"
Oct 30 22:10:36.697972 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:36.696943    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2" event={"ID":"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec","Type":"ContainerStarted","Data":"4dc7b716daba4e00a53f2febc9bafc3220bdd3af72edc5c0acabfcc351d3cabb"}
Oct 30 22:10:36.697972 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:36.697046    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2" event={"ID":"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec","Type":"ContainerStarted","Data":"9e5e93eb3b38d5e421bed4cbd85656bd95d8bbbe7ccc4e19e600996e9bfca7a3"}
Oct 30 22:10:36.700140 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:36.700098    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj" event={"ID":"fd6f1baa-ab90-488d-abe5-e70b4f6298ce","Type":"ContainerStarted","Data":"3d0c97ba29ffb11d992dbcdaf977a73a698ed2d410a78d19bc6cc036b07bdbd5"}
Oct 30 22:10:36.700325 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:36.700171    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj" event={"ID":"fd6f1baa-ab90-488d-abe5-e70b4f6298ce","Type":"ContainerStarted","Data":"014bfbc48312a70ef299b38bf05ade33d17794c6bd4217276d9b88bd1da2ea0f"}
Oct 30 22:10:36.724262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:36.708013    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg" event={"ID":"fa8f3772-b525-4512-a629-603102f076c0","Type":"ContainerStarted","Data":"7f7ead720820a6483861370e167447f3ece5c23128c2ad80b4702c27cf8149f7"}
Oct 30 22:10:36.724262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:36.708093    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg" event={"ID":"fa8f3772-b525-4512-a629-603102f076c0","Type":"ContainerStarted","Data":"ddad1ddd23d08b26ebfd6fbca87f14bab367732c94cdcfc448ff95378a7d7ae6"}
Oct 30 22:10:36.895266 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:36.895089    4871 patch_prober.go:29] interesting pod/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused" start-of-body=
Oct 30 22:10:36.895266 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:36.895166    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh" podUID="088f4439-453d-406a-b54a-837e10b36efd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused"
Oct 30 22:10:36.957325 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:36.950910    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2" podStartSLOduration=2.95088713 podStartE2EDuration="2.95088713s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:10:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:10:36.822614438 +0000 UTC m=+1979.000103056" watchObservedRunningTime="2026-10-30 22:10:36.95088713 +0000 UTC m=+1979.128375772"
Oct 30 22:10:37.084264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:37.083080    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg" podStartSLOduration=3.083060986 podStartE2EDuration="3.083060986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:10:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:10:37.081909059 +0000 UTC m=+1979.259397676" watchObservedRunningTime="2026-10-30 22:10:37.083060986 +0000 UTC m=+1979.260549606"
Oct 30 22:10:37.088171 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:37.088104    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj" podStartSLOduration=3.088081941 podStartE2EDuration="3.088081941s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:10:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:10:36.953826407 +0000 UTC m=+1979.131315028" watchObservedRunningTime="2026-10-30 22:10:37.088081941 +0000 UTC m=+1979.265570560"
Oct 30 22:10:37.292324 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:37.291349    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"
Oct 30 22:10:37.292324 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:37.291587    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-85wfj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.135:15021/healthz/ready\": dial tcp 10.133.0.135:15021: connect: connection refused" start-of-body=
Oct 30 22:10:37.292324 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:37.291647    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj" podUID="fd6f1baa-ab90-488d-abe5-e70b4f6298ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.135:15021/healthz/ready\": dial tcp 10.133.0.135:15021: connect: connection refused"
Oct 30 22:10:37.395263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:37.394652    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"
Oct 30 22:10:37.396282 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:37.396142    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-jbljg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.137:15021/healthz/ready\": dial tcp 10.133.0.137:15021: connect: connection refused" start-of-body=
Oct 30 22:10:37.396282 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:37.396201    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg" podUID="fa8f3772-b525-4512-a629-603102f076c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.137:15021/healthz/ready\": dial tcp 10.133.0.137:15021: connect: connection refused"
Oct 30 22:10:37.396282 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:37.396288    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"
Oct 30 22:10:37.396548 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:37.396324    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-2cvb2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused" start-of-body=
Oct 30 22:10:37.396548 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:37.396350    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2" podUID="9f13966e-b7ac-4a97-89b0-f98a9b44c5ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused"
Oct 30 22:10:37.883271 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:37.873735    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:38.700212 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:38.699190    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-2cvb2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused" start-of-body=
Oct 30 22:10:38.700212 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:38.699467    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2" podUID="9f13966e-b7ac-4a97-89b0-f98a9b44c5ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused"
Oct 30 22:10:38.700212 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:38.700185    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-jbljg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.137:15021/healthz/ready\": dial tcp 10.133.0.137:15021: connect: connection refused" start-of-body=
Oct 30 22:10:38.702092 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:38.700225    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg" podUID="fa8f3772-b525-4512-a629-603102f076c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.137:15021/healthz/ready\": dial tcp 10.133.0.137:15021: connect: connection refused"
Oct 30 22:10:38.702092 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:38.700758    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-85wfj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.135:15021/healthz/ready\": dial tcp 10.133.0.135:15021: connect: connection refused" start-of-body=
Oct 30 22:10:38.702092 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:38.700789    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj" podUID="fd6f1baa-ab90-488d-abe5-e70b4f6298ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.135:15021/healthz/ready\": dial tcp 10.133.0.135:15021: connect: connection refused"
Oct 30 22:10:38.784717 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:38.784676    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:38.801657 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:38.801622    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"
Oct 30 22:10:38.830258 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:38.805601    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"
Oct 30 22:10:38.892647 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:38.892612    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"
Oct 30 22:10:39.037381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:39.037295    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:39.219252 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:39.219202    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:39.303448 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:39.294731    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-85wfj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.135:15021/healthz/ready\": dial tcp 10.133.0.135:15021: connect: connection refused" start-of-body=
Oct 30 22:10:39.303448 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:39.294807    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj" podUID="fd6f1baa-ab90-488d-abe5-e70b4f6298ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.135:15021/healthz/ready\": dial tcp 10.133.0.135:15021: connect: connection refused"
Oct 30 22:10:39.396423 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:39.396390    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-2cvb2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused" start-of-body=
Oct 30 22:10:39.396695 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:39.396666    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2" podUID="9f13966e-b7ac-4a97-89b0-f98a9b44c5ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused"
Oct 30 22:10:39.545128 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:39.545029    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:39.609383 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:39.606467    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-jbljg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.137:15021/healthz/ready\": dial tcp 10.133.0.137:15021: connect: connection refused" start-of-body=
Oct 30 22:10:39.609383 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:39.606529    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg" podUID="fa8f3772-b525-4512-a629-603102f076c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.137:15021/healthz/ready\": dial tcp 10.133.0.137:15021: connect: connection refused"
Oct 30 22:10:39.647149 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:39.647118    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"]
Oct 30 22:10:39.853380 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:39.853352    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"
Oct 30 22:10:39.881266 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:39.867388    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:39.916636 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:39.915808    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-a3aa65-openshift-default-dockercfg-p6hb2\""
Oct 30 22:10:40.003263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:39.966638    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96df7172-8d94-4726-997e-2b0c9aeb8f5d-istiod-ca-cert\") pod \"e2e-api-b-a3aa65-openshift-default-57649d9c-657f7\" (UID: \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\") " pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"
Oct 30 22:10:40.003263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:39.966696    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96df7172-8d94-4726-997e-2b0c9aeb8f5d-istio-envoy\") pod \"e2e-api-b-a3aa65-openshift-default-57649d9c-657f7\" (UID: \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\") " pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"
Oct 30 22:10:40.003263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:39.966935    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96df7172-8d94-4726-997e-2b0c9aeb8f5d-workload-certs\") pod \"e2e-api-b-a3aa65-openshift-default-57649d9c-657f7\" (UID: \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\") " pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"
Oct 30 22:10:40.003263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:39.966968    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trqg9\" (UniqueName: \"kubernetes.io/projected/96df7172-8d94-4726-997e-2b0c9aeb8f5d-kube-api-access-trqg9\") pod \"e2e-api-b-a3aa65-openshift-default-57649d9c-657f7\" (UID: \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\") " pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"
Oct 30 22:10:40.003263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:39.966999    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96df7172-8d94-4726-997e-2b0c9aeb8f5d-istio-token\") pod \"e2e-api-b-a3aa65-openshift-default-57649d9c-657f7\" (UID: \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\") " pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"
Oct 30 22:10:40.003263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:39.967054    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96df7172-8d94-4726-997e-2b0c9aeb8f5d-istio-data\") pod \"e2e-api-b-a3aa65-openshift-default-57649d9c-657f7\" (UID: \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\") " pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"
Oct 30 22:10:40.003263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:39.967089    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96df7172-8d94-4726-997e-2b0c9aeb8f5d-istio-podinfo\") pod \"e2e-api-b-a3aa65-openshift-default-57649d9c-657f7\" (UID: \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\") " pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"
Oct 30 22:10:40.003263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:39.967232    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96df7172-8d94-4726-997e-2b0c9aeb8f5d-workload-socket\") pod \"e2e-api-b-a3aa65-openshift-default-57649d9c-657f7\" (UID: \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\") " pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"
Oct 30 22:10:40.003263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:39.967306    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96df7172-8d94-4726-997e-2b0c9aeb8f5d-credential-socket\") pod \"e2e-api-b-a3aa65-openshift-default-57649d9c-657f7\" (UID: \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\") " pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"
Oct 30 22:10:40.089285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:40.085577    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96df7172-8d94-4726-997e-2b0c9aeb8f5d-workload-socket\") pod \"e2e-api-b-a3aa65-openshift-default-57649d9c-657f7\" (UID: \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\") " pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"
Oct 30 22:10:40.089285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:40.085682    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96df7172-8d94-4726-997e-2b0c9aeb8f5d-credential-socket\") pod \"e2e-api-b-a3aa65-openshift-default-57649d9c-657f7\" (UID: \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\") " pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"
Oct 30 22:10:40.089285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:40.085849    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96df7172-8d94-4726-997e-2b0c9aeb8f5d-istiod-ca-cert\") pod \"e2e-api-b-a3aa65-openshift-default-57649d9c-657f7\" (UID: \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\") " pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"
Oct 30 22:10:40.089285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:40.085888    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96df7172-8d94-4726-997e-2b0c9aeb8f5d-istio-envoy\") pod \"e2e-api-b-a3aa65-openshift-default-57649d9c-657f7\" (UID: \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\") " pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"
Oct 30 22:10:40.089285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:40.085989    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96df7172-8d94-4726-997e-2b0c9aeb8f5d-workload-certs\") pod \"e2e-api-b-a3aa65-openshift-default-57649d9c-657f7\" (UID: \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\") " pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"
Oct 30 22:10:40.089285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:40.086015    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trqg9\" (UniqueName: \"kubernetes.io/projected/96df7172-8d94-4726-997e-2b0c9aeb8f5d-kube-api-access-trqg9\") pod \"e2e-api-b-a3aa65-openshift-default-57649d9c-657f7\" (UID: \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\") " pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"
Oct 30 22:10:40.089285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:40.086044    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96df7172-8d94-4726-997e-2b0c9aeb8f5d-istio-token\") pod \"e2e-api-b-a3aa65-openshift-default-57649d9c-657f7\" (UID: \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\") " pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"
Oct 30 22:10:40.089285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:40.086082    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96df7172-8d94-4726-997e-2b0c9aeb8f5d-istio-data\") pod \"e2e-api-b-a3aa65-openshift-default-57649d9c-657f7\" (UID: \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\") " pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"
Oct 30 22:10:40.089285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:40.086117    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96df7172-8d94-4726-997e-2b0c9aeb8f5d-istio-podinfo\") pod \"e2e-api-b-a3aa65-openshift-default-57649d9c-657f7\" (UID: \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\") " pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"
Oct 30 22:10:40.113732 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:40.094888    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96df7172-8d94-4726-997e-2b0c9aeb8f5d-workload-certs\") pod \"e2e-api-b-a3aa65-openshift-default-57649d9c-657f7\" (UID: \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\") " pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"
Oct 30 22:10:40.113732 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:40.095147    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96df7172-8d94-4726-997e-2b0c9aeb8f5d-credential-socket\") pod \"e2e-api-b-a3aa65-openshift-default-57649d9c-657f7\" (UID: \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\") " pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"
Oct 30 22:10:40.113732 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:40.095423    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96df7172-8d94-4726-997e-2b0c9aeb8f5d-workload-socket\") pod \"e2e-api-b-a3aa65-openshift-default-57649d9c-657f7\" (UID: \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\") " pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"
Oct 30 22:10:40.113732 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:40.096283    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96df7172-8d94-4726-997e-2b0c9aeb8f5d-istiod-ca-cert\") pod \"e2e-api-b-a3aa65-openshift-default-57649d9c-657f7\" (UID: \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\") " pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"
Oct 30 22:10:40.113732 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:40.096524    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96df7172-8d94-4726-997e-2b0c9aeb8f5d-istio-data\") pod \"e2e-api-b-a3aa65-openshift-default-57649d9c-657f7\" (UID: \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\") " pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"
Oct 30 22:10:40.183316 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:40.174009    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:40.194510 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:40.194009    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96df7172-8d94-4726-997e-2b0c9aeb8f5d-istio-podinfo\") pod \"e2e-api-b-a3aa65-openshift-default-57649d9c-657f7\" (UID: \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\") " pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"
Oct 30 22:10:40.195834 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:40.195804    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96df7172-8d94-4726-997e-2b0c9aeb8f5d-istio-envoy\") pod \"e2e-api-b-a3aa65-openshift-default-57649d9c-657f7\" (UID: \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\") " pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"
Oct 30 22:10:40.224539 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:40.198728    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"]
Oct 30 22:10:40.224539 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:40.199958    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-trqg9\" (UniqueName: \"kubernetes.io/projected/96df7172-8d94-4726-997e-2b0c9aeb8f5d-kube-api-access-trqg9\") pod \"e2e-api-b-a3aa65-openshift-default-57649d9c-657f7\" (UID: \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\") " pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"
Oct 30 22:10:40.224539 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:40.202410    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96df7172-8d94-4726-997e-2b0c9aeb8f5d-istio-token\") pod \"e2e-api-b-a3aa65-openshift-default-57649d9c-657f7\" (UID: \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\") " pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"
Oct 30 22:10:40.290298 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:40.289915    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-85wfj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.135:15021/healthz/ready\": dial tcp 10.133.0.135:15021: connect: connection refused" start-of-body=
Oct 30 22:10:40.290298 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:40.289998    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj" podUID="fd6f1baa-ab90-488d-abe5-e70b4f6298ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.135:15021/healthz/ready\": dial tcp 10.133.0.135:15021: connect: connection refused"
Oct 30 22:10:40.345262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:40.342497    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"
Oct 30 22:10:40.386827 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:40.386789    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-2cvb2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused" start-of-body=
Oct 30 22:10:40.387100 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:40.387071    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2" podUID="9f13966e-b7ac-4a97-89b0-f98a9b44c5ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused"
Oct 30 22:10:40.395953 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:40.395932    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-jbljg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.137:15021/healthz/ready\": dial tcp 10.133.0.137:15021: connect: connection refused" start-of-body=
Oct 30 22:10:40.396100 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:40.396078    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg" podUID="fa8f3772-b525-4512-a629-603102f076c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.137:15021/healthz/ready\": dial tcp 10.133.0.137:15021: connect: connection refused"
Oct 30 22:10:41.195192 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:10:41.195144    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96df7172_8d94_4726_997e_2b0c9aeb8f5d.slice/crio-3fb5b372ab24600b7d9d4261ee001801e36106225e821d7f72805012780ef5d3 WatchSource:0}: Error finding container 3fb5b372ab24600b7d9d4261ee001801e36106225e821d7f72805012780ef5d3: Status 404 returned error can't find the container with id 3fb5b372ab24600b7d9d4261ee001801e36106225e821d7f72805012780ef5d3
Oct 30 22:10:41.200035 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:41.199920    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:10:41.200035 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:41.200007    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"]
Oct 30 22:10:41.200211 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:41.200008    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:10:41.200211 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:41.200128    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:10:41.282735 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:41.282685    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-85wfj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.135:15021/healthz/ready\": dial tcp 10.133.0.135:15021: connect: connection refused" start-of-body=
Oct 30 22:10:41.282953 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:41.282768    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj" podUID="fd6f1baa-ab90-488d-abe5-e70b4f6298ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.135:15021/healthz/ready\": dial tcp 10.133.0.135:15021: connect: connection refused"
Oct 30 22:10:41.380211 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:41.379632    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-2cvb2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused" start-of-body=
Oct 30 22:10:41.380211 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:41.379697    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2" podUID="9f13966e-b7ac-4a97-89b0-f98a9b44c5ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused"
Oct 30 22:10:41.395506 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:41.395470    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-jbljg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.137:15021/healthz/ready\": dial tcp 10.133.0.137:15021: connect: connection refused" start-of-body=
Oct 30 22:10:41.395739 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:41.395710    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg" podUID="fa8f3772-b525-4512-a629-603102f076c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.137:15021/healthz/ready\": dial tcp 10.133.0.137:15021: connect: connection refused"
Oct 30 22:10:42.024677 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:42.024631    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7" event={"ID":"96df7172-8d94-4726-997e-2b0c9aeb8f5d","Type":"ContainerStarted","Data":"7d568e31411f948f4cf9680acd726967cd4fb7ebbcb98db95476ec161f5e201c"}
Oct 30 22:10:42.026016 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:42.025987    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7" event={"ID":"96df7172-8d94-4726-997e-2b0c9aeb8f5d","Type":"ContainerStarted","Data":"3fb5b372ab24600b7d9d4261ee001801e36106225e821d7f72805012780ef5d3"}
Oct 30 22:10:42.088858 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:42.088799    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:42.160263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:42.155595    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7" podStartSLOduration=3.155569166 podStartE2EDuration="3.155569166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:10:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:10:42.150964302 +0000 UTC m=+1984.328452920" watchObservedRunningTime="2026-10-30 22:10:42.155569166 +0000 UTC m=+1984.333057784"
Oct 30 22:10:42.255940 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:42.255903    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:42.291732 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:42.291648    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-85wfj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.135:15021/healthz/ready\": dial tcp 10.133.0.135:15021: connect: connection refused" start-of-body=
Oct 30 22:10:42.292017 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:42.291984    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj" podUID="fd6f1baa-ab90-488d-abe5-e70b4f6298ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.135:15021/healthz/ready\": dial tcp 10.133.0.135:15021: connect: connection refused"
Oct 30 22:10:42.348828 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:42.348794    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"
Oct 30 22:10:42.356972 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:42.356936    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"
Oct 30 22:10:42.380701 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:42.380658    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-2cvb2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused" start-of-body=
Oct 30 22:10:42.380993 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:42.380962    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2" podUID="9f13966e-b7ac-4a97-89b0-f98a9b44c5ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused"
Oct 30 22:10:42.412645 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:42.412608    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-jbljg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.137:15021/healthz/ready\": dial tcp 10.133.0.137:15021: connect: connection refused" start-of-body=
Oct 30 22:10:42.412937 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:42.412910    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg" podUID="fa8f3772-b525-4512-a629-603102f076c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.137:15021/healthz/ready\": dial tcp 10.133.0.137:15021: connect: connection refused"
Oct 30 22:10:43.047206 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:43.047172    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"
Oct 30 22:10:43.048547 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:43.048514    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"
Oct 30 22:10:43.286055 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:43.285160    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-85wfj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.135:15021/healthz/ready\": dial tcp 10.133.0.135:15021: connect: connection refused" start-of-body=
Oct 30 22:10:43.286055 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:43.285222    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj" podUID="fd6f1baa-ab90-488d-abe5-e70b4f6298ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.135:15021/healthz/ready\": dial tcp 10.133.0.135:15021: connect: connection refused"
Oct 30 22:10:43.379791 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:43.379746    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-2cvb2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused" start-of-body=
Oct 30 22:10:43.380004 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:43.379818    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2" podUID="9f13966e-b7ac-4a97-89b0-f98a9b44c5ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused"
Oct 30 22:10:43.394916 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:43.394881    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-jbljg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.137:15021/healthz/ready\": dial tcp 10.133.0.137:15021: connect: connection refused" start-of-body=
Oct 30 22:10:43.395067 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:43.394944    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg" podUID="fa8f3772-b525-4512-a629-603102f076c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.137:15021/healthz/ready\": dial tcp 10.133.0.137:15021: connect: connection refused"
Oct 30 22:10:43.563680 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:43.563643    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:43.949802 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:43.949760    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:44.105256 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:44.105215    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:44.281535 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:44.281431    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-85wfj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.135:15021/healthz/ready\": dial tcp 10.133.0.135:15021: connect: connection refused" start-of-body=
Oct 30 22:10:44.281535 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:44.281509    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj" podUID="fd6f1baa-ab90-488d-abe5-e70b4f6298ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.135:15021/healthz/ready\": dial tcp 10.133.0.135:15021: connect: connection refused"
Oct 30 22:10:44.380122 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:44.380084    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-2cvb2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused" start-of-body=
Oct 30 22:10:44.380690 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:44.380143    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2" podUID="9f13966e-b7ac-4a97-89b0-f98a9b44c5ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused"
Oct 30 22:10:44.394714 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:44.394684    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-jbljg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.137:15021/healthz/ready\": dial tcp 10.133.0.137:15021: connect: connection refused" start-of-body=
Oct 30 22:10:44.394909 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:44.394732    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg" podUID="fa8f3772-b525-4512-a629-603102f076c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.137:15021/healthz/ready\": dial tcp 10.133.0.137:15021: connect: connection refused"
Oct 30 22:10:44.852631 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:44.852593    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:45.018869 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:45.018835    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:45.174795 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:45.174758    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:46.259751 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:46.259715    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"
Oct 30 22:10:46.260725 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:46.260698    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"
Oct 30 22:10:46.261630 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:46.261607    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"
Oct 30 22:10:46.280994 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:46.280899    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-85wfj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.135:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:10:46.280994 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:46.280968    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj" podUID="fd6f1baa-ab90-488d-abe5-e70b4f6298ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.135:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:10:46.303658 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:46.303626    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"
Oct 30 22:10:46.304605 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:46.304581    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"
Oct 30 22:10:46.305808 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:46.305785    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"
Oct 30 22:10:46.395415 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:46.395380    4871 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7df854bf67-jbljg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.137:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:10:46.395637 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:46.395452    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg" podUID="fa8f3772-b525-4512-a629-603102f076c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.137:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:10:46.503887 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:46.503842    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"
Oct 30 22:10:46.504306 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:46.504280    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"
Oct 30 22:10:46.505165 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:46.505143    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"
Oct 30 22:10:47.902954 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:47.902913    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"]
Oct 30 22:10:47.903354 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:47.903167    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x" podUID="a43db63c-f279-45cb-9096-22b48545b36a" containerName="istio-proxy" containerID="cri-o://462cf8e9e0e84e730a315ff3d76e2d608c2f769bbfcfdb9624069be66290ff53" gracePeriod=30
Oct 30 22:10:48.113487 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:48.113417    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:48.351199 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:48.351027    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:48.952661 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:48.952625    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:52.931659 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:52.931621    4871 patch_prober.go:29] interesting pod/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.110:15021/healthz/ready\": dial tcp 10.133.0.110:15021: connect: connection refused" start-of-body=
Oct 30 22:10:52.932122 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:52.931694    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x" podUID="a43db63c-f279-45cb-9096-22b48545b36a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.110:15021/healthz/ready\": dial tcp 10.133.0.110:15021: connect: connection refused"
Oct 30 22:10:53.169185 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.169139    4871 generic.go:356] "Generic (PLEG): container finished" podID="a43db63c-f279-45cb-9096-22b48545b36a" containerID="462cf8e9e0e84e730a315ff3d76e2d608c2f769bbfcfdb9624069be66290ff53" exitCode=0
Oct 30 22:10:53.169380 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.169191    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x" event={"ID":"a43db63c-f279-45cb-9096-22b48545b36a","Type":"ContainerDied","Data":"462cf8e9e0e84e730a315ff3d76e2d608c2f769bbfcfdb9624069be66290ff53"}
Oct 30 22:10:53.203059 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.203029    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"
Oct 30 22:10:53.262595 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.262548    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a43db63c-f279-45cb-9096-22b48545b36a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a43db63c-f279-45cb-9096-22b48545b36a-workload-socket\") pod \"a43db63c-f279-45cb-9096-22b48545b36a\" (UID: \"a43db63c-f279-45cb-9096-22b48545b36a\") "
Oct 30 22:10:53.262817 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.262754    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a43db63c-f279-45cb-9096-22b48545b36a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a43db63c-f279-45cb-9096-22b48545b36a-istio-podinfo\") pod \"a43db63c-f279-45cb-9096-22b48545b36a\" (UID: \"a43db63c-f279-45cb-9096-22b48545b36a\") "
Oct 30 22:10:53.262896 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.262847    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a43db63c-f279-45cb-9096-22b48545b36a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a43db63c-f279-45cb-9096-22b48545b36a-istiod-ca-cert\") pod \"a43db63c-f279-45cb-9096-22b48545b36a\" (UID: \"a43db63c-f279-45cb-9096-22b48545b36a\") "
Oct 30 22:10:53.262896 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.262879    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a43db63c-f279-45cb-9096-22b48545b36a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a43db63c-f279-45cb-9096-22b48545b36a-istio-envoy\") pod \"a43db63c-f279-45cb-9096-22b48545b36a\" (UID: \"a43db63c-f279-45cb-9096-22b48545b36a\") "
Oct 30 22:10:53.262996 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.262889    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a43db63c-f279-45cb-9096-22b48545b36a-workload-socket" pod "a43db63c-f279-45cb-9096-22b48545b36a" (UID: "a43db63c-f279-45cb-9096-22b48545b36a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:53.262996 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.262954    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a43db63c-f279-45cb-9096-22b48545b36a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a43db63c-f279-45cb-9096-22b48545b36a-workload-certs\") pod \"a43db63c-f279-45cb-9096-22b48545b36a\" (UID: \"a43db63c-f279-45cb-9096-22b48545b36a\") "
Oct 30 22:10:53.262996 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.262985    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a43db63c-f279-45cb-9096-22b48545b36a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a43db63c-f279-45cb-9096-22b48545b36a-istio-data\") pod \"a43db63c-f279-45cb-9096-22b48545b36a\" (UID: \"a43db63c-f279-45cb-9096-22b48545b36a\") "
Oct 30 22:10:53.263150 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.263035    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a43db63c-f279-45cb-9096-22b48545b36a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a43db63c-f279-45cb-9096-22b48545b36a-credential-socket\") pod \"a43db63c-f279-45cb-9096-22b48545b36a\" (UID: \"a43db63c-f279-45cb-9096-22b48545b36a\") "
Oct 30 22:10:53.263274 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.263156    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a43db63c-f279-45cb-9096-22b48545b36a-kube-api-access-r5w5q\" (UniqueName: \"kubernetes.io/projected/a43db63c-f279-45cb-9096-22b48545b36a-kube-api-access-r5w5q\") pod \"a43db63c-f279-45cb-9096-22b48545b36a\" (UID: \"a43db63c-f279-45cb-9096-22b48545b36a\") "
Oct 30 22:10:53.263398 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.263307    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a43db63c-f279-45cb-9096-22b48545b36a-istiod-ca-cert" pod "a43db63c-f279-45cb-9096-22b48545b36a" (UID: "a43db63c-f279-45cb-9096-22b48545b36a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:53.263495 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.263464    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a43db63c-f279-45cb-9096-22b48545b36a-istio-data" pod "a43db63c-f279-45cb-9096-22b48545b36a" (UID: "a43db63c-f279-45cb-9096-22b48545b36a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:53.263495 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.263481    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a43db63c-f279-45cb-9096-22b48545b36a-credential-socket" pod "a43db63c-f279-45cb-9096-22b48545b36a" (UID: "a43db63c-f279-45cb-9096-22b48545b36a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:53.263689 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.263502    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a43db63c-f279-45cb-9096-22b48545b36a-istio-token\" (UniqueName: \"kubernetes.io/projected/a43db63c-f279-45cb-9096-22b48545b36a-istio-token\") pod \"a43db63c-f279-45cb-9096-22b48545b36a\" (UID: \"a43db63c-f279-45cb-9096-22b48545b36a\") "
Oct 30 22:10:53.263775 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.263733    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a43db63c-f279-45cb-9096-22b48545b36a-workload-certs" pod "a43db63c-f279-45cb-9096-22b48545b36a" (UID: "a43db63c-f279-45cb-9096-22b48545b36a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:53.264538 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.264380    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a43db63c-f279-45cb-9096-22b48545b36a-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:10:53.264538 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.264411    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a43db63c-f279-45cb-9096-22b48545b36a-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:10:53.264538 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.264426    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a43db63c-f279-45cb-9096-22b48545b36a-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:10:53.264538 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.264442    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a43db63c-f279-45cb-9096-22b48545b36a-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:10:53.264538 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.264457    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a43db63c-f279-45cb-9096-22b48545b36a-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:10:53.265448 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.265419    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a43db63c-f279-45cb-9096-22b48545b36a-istio-envoy" pod "a43db63c-f279-45cb-9096-22b48545b36a" (UID: "a43db63c-f279-45cb-9096-22b48545b36a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:53.265711 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.265687    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a43db63c-f279-45cb-9096-22b48545b36a-kube-api-access-r5w5q" pod "a43db63c-f279-45cb-9096-22b48545b36a" (UID: "a43db63c-f279-45cb-9096-22b48545b36a"). InnerVolumeSpecName "kube-api-access-r5w5q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:53.265870 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.265843    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a43db63c-f279-45cb-9096-22b48545b36a-istio-podinfo" pod "a43db63c-f279-45cb-9096-22b48545b36a" (UID: "a43db63c-f279-45cb-9096-22b48545b36a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:10:53.265971 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.265949    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a43db63c-f279-45cb-9096-22b48545b36a-istio-token" pod "a43db63c-f279-45cb-9096-22b48545b36a" (UID: "a43db63c-f279-45cb-9096-22b48545b36a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:53.304424 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.304380    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj"]
Oct 30 22:10:53.305057 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.305038    4871 cpu_manager.go:414] "containerMap: removing container" podUID="a43db63c-f279-45cb-9096-22b48545b36a" containerName="istio-proxy"
Oct 30 22:10:53.305057 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.305057    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a43db63c-f279-45cb-9096-22b48545b36a" containerName="istio-proxy"
Oct 30 22:10:53.305213 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.305198    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="a43db63c-f279-45cb-9096-22b48545b36a" containerName="istio-proxy"
Oct 30 22:10:53.309106 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.309080    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj"
Oct 30 22:10:53.329514 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.329469    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj"]
Oct 30 22:10:53.358947 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.358912    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-f0b8c9-serving-cert\""
Oct 30 22:10:53.365756 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.365719    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4669d\" (UniqueName: \"kubernetes.io/projected/3df5f9f5-7977-43db-be67-e1105d82e626-kube-api-access-4669d\") pod \"maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj\" (UID: \"3df5f9f5-7977-43db-be67-e1105d82e626\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj"
Oct 30 22:10:53.365941 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.365874    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3df5f9f5-7977-43db-be67-e1105d82e626-maas-api-tls\") pod \"maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj\" (UID: \"3df5f9f5-7977-43db-be67-e1105d82e626\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj"
Oct 30 22:10:53.365941 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.365917    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3df5f9f5-7977-43db-be67-e1105d82e626-metrics-certs\") pod \"maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj\" (UID: \"3df5f9f5-7977-43db-be67-e1105d82e626\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj"
Oct 30 22:10:53.366102 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.366056    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3df5f9f5-7977-43db-be67-e1105d82e626-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj\" (UID: \"3df5f9f5-7977-43db-be67-e1105d82e626\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj"
Oct 30 22:10:53.366344 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.366324    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a43db63c-f279-45cb-9096-22b48545b36a-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:10:53.366344 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.366345    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a43db63c-f279-45cb-9096-22b48545b36a-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:10:53.366499 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.366356    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5w5q\" (UniqueName: \"kubernetes.io/projected/a43db63c-f279-45cb-9096-22b48545b36a-kube-api-access-r5w5q\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:10:53.366499 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.366371    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a43db63c-f279-45cb-9096-22b48545b36a-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:10:53.457910 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.457828    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:53.467743 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.467708    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3df5f9f5-7977-43db-be67-e1105d82e626-maas-api-tls\") pod \"maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj\" (UID: \"3df5f9f5-7977-43db-be67-e1105d82e626\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj"
Oct 30 22:10:53.467743 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.467744    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3df5f9f5-7977-43db-be67-e1105d82e626-metrics-certs\") pod \"maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj\" (UID: \"3df5f9f5-7977-43db-be67-e1105d82e626\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj"
Oct 30 22:10:53.467964 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.467786    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3df5f9f5-7977-43db-be67-e1105d82e626-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj\" (UID: \"3df5f9f5-7977-43db-be67-e1105d82e626\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj"
Oct 30 22:10:53.467964 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:10:53.467879    4871 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0b8c9-serving-cert: secret "maas-api-e2e-ipp-a-f0b8c9-serving-cert" not found
Oct 30 22:10:53.467964 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.467921    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4669d\" (UniqueName: \"kubernetes.io/projected/3df5f9f5-7977-43db-be67-e1105d82e626-kube-api-access-4669d\") pod \"maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj\" (UID: \"3df5f9f5-7977-43db-be67-e1105d82e626\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj"
Oct 30 22:10:53.468082 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:10:53.467966    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3df5f9f5-7977-43db-be67-e1105d82e626-maas-api-tls podName:3df5f9f5-7977-43db-be67-e1105d82e626 nodeName:}" failed. No retries permitted until 2026-10-30 22:10:53.967944723 +0000 UTC m=+1996.145433333 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3df5f9f5-7977-43db-be67-e1105d82e626-maas-api-tls") pod "maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj" (UID: "3df5f9f5-7977-43db-be67-e1105d82e626") : secret "maas-api-e2e-ipp-a-f0b8c9-serving-cert" not found
Oct 30 22:10:53.468597 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.468565    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3df5f9f5-7977-43db-be67-e1105d82e626-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj\" (UID: \"3df5f9f5-7977-43db-be67-e1105d82e626\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj"
Oct 30 22:10:53.470350 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.470331    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3df5f9f5-7977-43db-be67-e1105d82e626-metrics-certs\") pod \"maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj\" (UID: \"3df5f9f5-7977-43db-be67-e1105d82e626\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj"
Oct 30 22:10:53.552076 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.552037    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4669d\" (UniqueName: \"kubernetes.io/projected/3df5f9f5-7977-43db-be67-e1105d82e626-kube-api-access-4669d\") pod \"maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj\" (UID: \"3df5f9f5-7977-43db-be67-e1105d82e626\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj"
Oct 30 22:10:53.614858 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.614810    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:53.786380 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.786153    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:53.977358 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.977329    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3df5f9f5-7977-43db-be67-e1105d82e626-maas-api-tls\") pod \"maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj\" (UID: \"3df5f9f5-7977-43db-be67-e1105d82e626\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj"
Oct 30 22:10:53.980037 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:53.980005    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3df5f9f5-7977-43db-be67-e1105d82e626-maas-api-tls\") pod \"maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj\" (UID: \"3df5f9f5-7977-43db-be67-e1105d82e626\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj"
Oct 30 22:10:54.179253 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:54.179200    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x" event={"ID":"a43db63c-f279-45cb-9096-22b48545b36a","Type":"ContainerDied","Data":"0492f6519bb515a454ce83d190b76435e45d9b16a12b1bf7e986d1860e6da8b3"}
Oct 30 22:10:54.179253 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:54.179230    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"
Oct 30 22:10:54.179562 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:54.179292    4871 scope.go:122] "RemoveContainer" containerID="462cf8e9e0e84e730a315ff3d76e2d608c2f769bbfcfdb9624069be66290ff53"
Oct 30 22:10:54.224723 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:54.224680    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj"
Oct 30 22:10:54.342480 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:54.342434    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"]
Oct 30 22:10:54.385945 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:54.385910    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-gv94x"]
Oct 30 22:10:54.604905 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:54.604868    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj"]
Oct 30 22:10:54.605055 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:10:54.604934    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3df5f9f5_7977_43db_be67_e1105d82e626.slice/crio-7f2ff8b7e1ed7d6f654b8369bdaf573b25444a462df301d33ed3617ec2e4dbaa WatchSource:0}: Error finding container 7f2ff8b7e1ed7d6f654b8369bdaf573b25444a462df301d33ed3617ec2e4dbaa: Status 404 returned error can't find the container with id 7f2ff8b7e1ed7d6f654b8369bdaf573b25444a462df301d33ed3617ec2e4dbaa
Oct 30 22:10:55.189270 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:55.189212    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj" event={"ID":"3df5f9f5-7977-43db-be67-e1105d82e626","Type":"ContainerStarted","Data":"7f2ff8b7e1ed7d6f654b8369bdaf573b25444a462df301d33ed3617ec2e4dbaa"}
Oct 30 22:10:55.961632 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:55.961601    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a43db63c-f279-45cb-9096-22b48545b36a" path="/var/lib/kubelet/pods/a43db63c-f279-45cb-9096-22b48545b36a/volumes"
Oct 30 22:10:56.278116 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:56.278021    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj" event={"ID":"3df5f9f5-7977-43db-be67-e1105d82e626","Type":"ContainerStarted","Data":"82763cfaca8b7c0bd3fe093bade43f098bbad998c56ae4b621c6d176798554a8"}
Oct 30 22:10:56.278613 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:56.278380    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj"
Oct 30 22:10:56.393206 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:56.393147    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj" podStartSLOduration=2.414172624 podStartE2EDuration="3.393130503s" totalImagesPullingTime="978.957879ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:10:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:10:56.38992499 +0000 UTC m=+1998.567413607" watchObservedRunningTime="2026-10-30 22:10:56.393130503 +0000 UTC m=+1998.570619119"
Oct 30 22:10:57.576483 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:57.576446    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t"]
Oct 30 22:10:57.576912 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:57.576648    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t" podUID="07f3e008-0adf-4515-b983-fba8739d5ad9" containerName="maas-api" containerID="cri-o://fada876b7a5b74ebadd4146050e517cfcc16d9e14d2c2b56f9fbf004112ad41c" gracePeriod=30
Oct 30 22:10:57.683136 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:57.683097    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:57.924434 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:57.924405    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t"
Oct 30 22:10:58.051262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:58.049136    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07f3e008-0adf-4515-b983-fba8739d5ad9-openshift-service-ca" pod "07f3e008-0adf-4515-b983-fba8739d5ad9" (UID: "07f3e008-0adf-4515-b983-fba8739d5ad9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:58.051262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:58.049703    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/07f3e008-0adf-4515-b983-fba8739d5ad9-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07f3e008-0adf-4515-b983-fba8739d5ad9-openshift-service-ca\") pod \"07f3e008-0adf-4515-b983-fba8739d5ad9\" (UID: \"07f3e008-0adf-4515-b983-fba8739d5ad9\") "
Oct 30 22:10:58.051262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:58.050655    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/07f3e008-0adf-4515-b983-fba8739d5ad9-metrics-certs\" (UniqueName: \"kubernetes.io/secret/07f3e008-0adf-4515-b983-fba8739d5ad9-metrics-certs\") pod \"07f3e008-0adf-4515-b983-fba8739d5ad9\" (UID: \"07f3e008-0adf-4515-b983-fba8739d5ad9\") "
Oct 30 22:10:58.051262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:58.050812    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/07f3e008-0adf-4515-b983-fba8739d5ad9-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07f3e008-0adf-4515-b983-fba8739d5ad9-maas-api-tls\") pod \"07f3e008-0adf-4515-b983-fba8739d5ad9\" (UID: \"07f3e008-0adf-4515-b983-fba8739d5ad9\") "
Oct 30 22:10:58.052221 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:58.052189    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/07f3e008-0adf-4515-b983-fba8739d5ad9-kube-api-access-h27k4\" (UniqueName: \"kubernetes.io/projected/07f3e008-0adf-4515-b983-fba8739d5ad9-kube-api-access-h27k4\") pod \"07f3e008-0adf-4515-b983-fba8739d5ad9\" (UID: \"07f3e008-0adf-4515-b983-fba8739d5ad9\") "
Oct 30 22:10:58.062319 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:58.054221    4871 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07f3e008-0adf-4515-b983-fba8739d5ad9-openshift-service-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:10:58.098157 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:58.098110    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07f3e008-0adf-4515-b983-fba8739d5ad9-maas-api-tls" pod "07f3e008-0adf-4515-b983-fba8739d5ad9" (UID: "07f3e008-0adf-4515-b983-fba8739d5ad9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:10:58.098331 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:58.098167    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07f3e008-0adf-4515-b983-fba8739d5ad9-metrics-certs" pod "07f3e008-0adf-4515-b983-fba8739d5ad9" (UID: "07f3e008-0adf-4515-b983-fba8739d5ad9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:10:58.101230 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:58.101196    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07f3e008-0adf-4515-b983-fba8739d5ad9-kube-api-access-h27k4" pod "07f3e008-0adf-4515-b983-fba8739d5ad9" (UID: "07f3e008-0adf-4515-b983-fba8739d5ad9"). InnerVolumeSpecName "kube-api-access-h27k4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:58.164673 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:58.164567    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07f3e008-0adf-4515-b983-fba8739d5ad9-maas-api-tls\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:10:58.164673 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:58.164601    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h27k4\" (UniqueName: \"kubernetes.io/projected/07f3e008-0adf-4515-b983-fba8739d5ad9-kube-api-access-h27k4\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:10:58.164673 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:58.164612    4871 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07f3e008-0adf-4515-b983-fba8739d5ad9-metrics-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:10:58.293639 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:58.293598    4871 generic.go:356] "Generic (PLEG): container finished" podID="07f3e008-0adf-4515-b983-fba8739d5ad9" containerID="fada876b7a5b74ebadd4146050e517cfcc16d9e14d2c2b56f9fbf004112ad41c" exitCode=0
Oct 30 22:10:58.293818 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:58.293656    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t"
Oct 30 22:10:58.293818 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:58.293686    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t" event={"ID":"07f3e008-0adf-4515-b983-fba8739d5ad9","Type":"ContainerDied","Data":"fada876b7a5b74ebadd4146050e517cfcc16d9e14d2c2b56f9fbf004112ad41c"}
Oct 30 22:10:58.293818 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:58.293745    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t" event={"ID":"07f3e008-0adf-4515-b983-fba8739d5ad9","Type":"ContainerDied","Data":"4f645bcddfdd11325f33d219bbef6d50aaf789e1dd085b951f1e936a8062aba6"}
Oct 30 22:10:58.293818 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:58.293767    4871 scope.go:122] "RemoveContainer" containerID="fada876b7a5b74ebadd4146050e517cfcc16d9e14d2c2b56f9fbf004112ad41c"
Oct 30 22:10:58.305836 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:58.305805    4871 scope.go:122] "RemoveContainer" containerID="fada876b7a5b74ebadd4146050e517cfcc16d9e14d2c2b56f9fbf004112ad41c"
Oct 30 22:10:58.306148 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:10:58.306128    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fada876b7a5b74ebadd4146050e517cfcc16d9e14d2c2b56f9fbf004112ad41c\": container with ID starting with fada876b7a5b74ebadd4146050e517cfcc16d9e14d2c2b56f9fbf004112ad41c not found: ID does not exist" containerID="fada876b7a5b74ebadd4146050e517cfcc16d9e14d2c2b56f9fbf004112ad41c"
Oct 30 22:10:58.306202 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:58.306156    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fada876b7a5b74ebadd4146050e517cfcc16d9e14d2c2b56f9fbf004112ad41c"} err="failed to get container status \"fada876b7a5b74ebadd4146050e517cfcc16d9e14d2c2b56f9fbf004112ad41c\": rpc error: code = NotFound desc = could not find container \"fada876b7a5b74ebadd4146050e517cfcc16d9e14d2c2b56f9fbf004112ad41c\": container with ID starting with fada876b7a5b74ebadd4146050e517cfcc16d9e14d2c2b56f9fbf004112ad41c not found: ID does not exist"
Oct 30 22:10:58.452855 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:58.452763    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t"]
Oct 30 22:10:58.457788 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:58.457763    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2f6bc1-dd55cdc74-dzk4t"]
Oct 30 22:10:59.188812 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:59.188777    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:59.415461 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:59.415418    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:59.569342 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:59.569232    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:59.968608 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:10:59.968561    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="07f3e008-0adf-4515-b983-fba8739d5ad9" path="/var/lib/kubelet/pods/07f3e008-0adf-4515-b983-fba8739d5ad9/volumes"
Oct 30 22:11:00.149914 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:00.149877    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:02.295511 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:02.295426    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj"
Oct 30 22:11:04.528155 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:04.528119    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:05.571515 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:11:05.571467    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:11:06.054776 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:06.054741    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:06.207427 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:06.207385    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:06.422850 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:06.422813    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"]
Oct 30 22:11:06.423106 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:06.423080    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh" podUID="42c86b8e-04b6-49fa-af33-9bc73297d162" containerName="istio-proxy" containerID="cri-o://0f3e66340d43693a333b872b1a92745111c02cd543488d0ed788eb3e199960a0" gracePeriod=30
Oct 30 22:11:06.430085 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:06.430042    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:09.105576 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:09.105539    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:11.475605 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:11:11.475570    4871 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42c86b8e_04b6_49fa_af33_9bc73297d162.slice/crio-0f3e66340d43693a333b872b1a92745111c02cd543488d0ed788eb3e199960a0.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 22:11:11.705442 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:11.705374    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"
Oct 30 22:11:11.821843 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:11.821795    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/42c86b8e-04b6-49fa-af33-9bc73297d162-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42c86b8e-04b6-49fa-af33-9bc73297d162-istio-podinfo\") pod \"42c86b8e-04b6-49fa-af33-9bc73297d162\" (UID: \"42c86b8e-04b6-49fa-af33-9bc73297d162\") "
Oct 30 22:11:11.822047 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:11.821862    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42c86b8e-04b6-49fa-af33-9bc73297d162-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42c86b8e-04b6-49fa-af33-9bc73297d162-istio-envoy\") pod \"42c86b8e-04b6-49fa-af33-9bc73297d162\" (UID: \"42c86b8e-04b6-49fa-af33-9bc73297d162\") "
Oct 30 22:11:11.822047 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:11.821997    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/42c86b8e-04b6-49fa-af33-9bc73297d162-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42c86b8e-04b6-49fa-af33-9bc73297d162-istiod-ca-cert\") pod \"42c86b8e-04b6-49fa-af33-9bc73297d162\" (UID: \"42c86b8e-04b6-49fa-af33-9bc73297d162\") "
Oct 30 22:11:11.822047 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:11.822034    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/42c86b8e-04b6-49fa-af33-9bc73297d162-kube-api-access-rhmlm\" (UniqueName: \"kubernetes.io/projected/42c86b8e-04b6-49fa-af33-9bc73297d162-kube-api-access-rhmlm\") pod \"42c86b8e-04b6-49fa-af33-9bc73297d162\" (UID: \"42c86b8e-04b6-49fa-af33-9bc73297d162\") "
Oct 30 22:11:11.822221 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:11.822061    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/42c86b8e-04b6-49fa-af33-9bc73297d162-istio-token\" (UniqueName: \"kubernetes.io/projected/42c86b8e-04b6-49fa-af33-9bc73297d162-istio-token\") pod \"42c86b8e-04b6-49fa-af33-9bc73297d162\" (UID: \"42c86b8e-04b6-49fa-af33-9bc73297d162\") "
Oct 30 22:11:11.822221 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:11.822116    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42c86b8e-04b6-49fa-af33-9bc73297d162-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42c86b8e-04b6-49fa-af33-9bc73297d162-istio-data\") pod \"42c86b8e-04b6-49fa-af33-9bc73297d162\" (UID: \"42c86b8e-04b6-49fa-af33-9bc73297d162\") "
Oct 30 22:11:11.822443 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:11.822412    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42c86b8e-04b6-49fa-af33-9bc73297d162-istiod-ca-cert" pod "42c86b8e-04b6-49fa-af33-9bc73297d162" (UID: "42c86b8e-04b6-49fa-af33-9bc73297d162"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:11:11.822522 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:11.822465    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42c86b8e-04b6-49fa-af33-9bc73297d162-istio-data" pod "42c86b8e-04b6-49fa-af33-9bc73297d162" (UID: "42c86b8e-04b6-49fa-af33-9bc73297d162"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:11.822769 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:11.822741    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42c86b8e-04b6-49fa-af33-9bc73297d162-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42c86b8e-04b6-49fa-af33-9bc73297d162-workload-socket\") pod \"42c86b8e-04b6-49fa-af33-9bc73297d162\" (UID: \"42c86b8e-04b6-49fa-af33-9bc73297d162\") "
Oct 30 22:11:11.823022 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:11.822995    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42c86b8e-04b6-49fa-af33-9bc73297d162-workload-socket" pod "42c86b8e-04b6-49fa-af33-9bc73297d162" (UID: "42c86b8e-04b6-49fa-af33-9bc73297d162"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:11.823130 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:11.823066    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42c86b8e-04b6-49fa-af33-9bc73297d162-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42c86b8e-04b6-49fa-af33-9bc73297d162-workload-certs\") pod \"42c86b8e-04b6-49fa-af33-9bc73297d162\" (UID: \"42c86b8e-04b6-49fa-af33-9bc73297d162\") "
Oct 30 22:11:11.823130 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:11.823108    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42c86b8e-04b6-49fa-af33-9bc73297d162-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42c86b8e-04b6-49fa-af33-9bc73297d162-credential-socket\") pod \"42c86b8e-04b6-49fa-af33-9bc73297d162\" (UID: \"42c86b8e-04b6-49fa-af33-9bc73297d162\") "
Oct 30 22:11:11.823587 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:11.823377    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42c86b8e-04b6-49fa-af33-9bc73297d162-workload-certs" pod "42c86b8e-04b6-49fa-af33-9bc73297d162" (UID: "42c86b8e-04b6-49fa-af33-9bc73297d162"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:11.823709 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:11.823607    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42c86b8e-04b6-49fa-af33-9bc73297d162-credential-socket" pod "42c86b8e-04b6-49fa-af33-9bc73297d162" (UID: "42c86b8e-04b6-49fa-af33-9bc73297d162"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:11.824104 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:11.824078    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42c86b8e-04b6-49fa-af33-9bc73297d162-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:11:11.824298 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:11.824109    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42c86b8e-04b6-49fa-af33-9bc73297d162-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:11:11.824298 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:11.824125    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42c86b8e-04b6-49fa-af33-9bc73297d162-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:11:11.824298 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:11.824139    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42c86b8e-04b6-49fa-af33-9bc73297d162-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:11:11.824298 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:11.824152    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42c86b8e-04b6-49fa-af33-9bc73297d162-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:11:11.824525 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:11.824448    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/42c86b8e-04b6-49fa-af33-9bc73297d162-istio-podinfo" pod "42c86b8e-04b6-49fa-af33-9bc73297d162" (UID: "42c86b8e-04b6-49fa-af33-9bc73297d162"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:11:11.824612 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:11.824578    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42c86b8e-04b6-49fa-af33-9bc73297d162-kube-api-access-rhmlm" pod "42c86b8e-04b6-49fa-af33-9bc73297d162" (UID: "42c86b8e-04b6-49fa-af33-9bc73297d162"). InnerVolumeSpecName "kube-api-access-rhmlm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:11.824718 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:11.824671    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42c86b8e-04b6-49fa-af33-9bc73297d162-istio-envoy" pod "42c86b8e-04b6-49fa-af33-9bc73297d162" (UID: "42c86b8e-04b6-49fa-af33-9bc73297d162"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:11.824718 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:11.824677    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42c86b8e-04b6-49fa-af33-9bc73297d162-istio-token" pod "42c86b8e-04b6-49fa-af33-9bc73297d162" (UID: "42c86b8e-04b6-49fa-af33-9bc73297d162"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:11.925611 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:11.925571    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rhmlm\" (UniqueName: \"kubernetes.io/projected/42c86b8e-04b6-49fa-af33-9bc73297d162-kube-api-access-rhmlm\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:11:11.925611 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:11.925606    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42c86b8e-04b6-49fa-af33-9bc73297d162-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:11:11.925611 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:11.925616    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42c86b8e-04b6-49fa-af33-9bc73297d162-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:11:11.925859 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:11.925626    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42c86b8e-04b6-49fa-af33-9bc73297d162-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:11:12.449897 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:12.449860    4871 generic.go:356] "Generic (PLEG): container finished" podID="42c86b8e-04b6-49fa-af33-9bc73297d162" containerID="0f3e66340d43693a333b872b1a92745111c02cd543488d0ed788eb3e199960a0" exitCode=0
Oct 30 22:11:12.450082 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:12.449905    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"
Oct 30 22:11:12.450082 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:12.449922    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh" event={"ID":"42c86b8e-04b6-49fa-af33-9bc73297d162","Type":"ContainerDied","Data":"0f3e66340d43693a333b872b1a92745111c02cd543488d0ed788eb3e199960a0"}
Oct 30 22:11:12.450082 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:12.449972    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh" event={"ID":"42c86b8e-04b6-49fa-af33-9bc73297d162","Type":"ContainerDied","Data":"7509a503aab9cb9721649dc7e0726795af5af3ca8d1a46091b8b434a0f18551d"}
Oct 30 22:11:12.450082 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:12.449995    4871 scope.go:122] "RemoveContainer" containerID="0f3e66340d43693a333b872b1a92745111c02cd543488d0ed788eb3e199960a0"
Oct 30 22:11:12.460849 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:12.460818    4871 scope.go:122] "RemoveContainer" containerID="0f027d9405f9db17345e77c83cb859288cbc11f8f45e5c9c2349f0487b3bcb21"
Oct 30 22:11:12.473613 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:12.473589    4871 scope.go:122] "RemoveContainer" containerID="0f3e66340d43693a333b872b1a92745111c02cd543488d0ed788eb3e199960a0"
Oct 30 22:11:12.473933 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:11:12.473915    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f3e66340d43693a333b872b1a92745111c02cd543488d0ed788eb3e199960a0\": container with ID starting with 0f3e66340d43693a333b872b1a92745111c02cd543488d0ed788eb3e199960a0 not found: ID does not exist" containerID="0f3e66340d43693a333b872b1a92745111c02cd543488d0ed788eb3e199960a0"
Oct 30 22:11:12.473990 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:12.473941    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f3e66340d43693a333b872b1a92745111c02cd543488d0ed788eb3e199960a0"} err="failed to get container status \"0f3e66340d43693a333b872b1a92745111c02cd543488d0ed788eb3e199960a0\": rpc error: code = NotFound desc = could not find container \"0f3e66340d43693a333b872b1a92745111c02cd543488d0ed788eb3e199960a0\": container with ID starting with 0f3e66340d43693a333b872b1a92745111c02cd543488d0ed788eb3e199960a0 not found: ID does not exist"
Oct 30 22:11:12.473990 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:12.473960    4871 scope.go:122] "RemoveContainer" containerID="0f027d9405f9db17345e77c83cb859288cbc11f8f45e5c9c2349f0487b3bcb21"
Oct 30 22:11:12.474224 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:11:12.474197    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f027d9405f9db17345e77c83cb859288cbc11f8f45e5c9c2349f0487b3bcb21\": container with ID starting with 0f027d9405f9db17345e77c83cb859288cbc11f8f45e5c9c2349f0487b3bcb21 not found: ID does not exist" containerID="0f027d9405f9db17345e77c83cb859288cbc11f8f45e5c9c2349f0487b3bcb21"
Oct 30 22:11:12.474224 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:12.474229    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f027d9405f9db17345e77c83cb859288cbc11f8f45e5c9c2349f0487b3bcb21"} err="failed to get container status \"0f027d9405f9db17345e77c83cb859288cbc11f8f45e5c9c2349f0487b3bcb21\": rpc error: code = NotFound desc = could not find container \"0f027d9405f9db17345e77c83cb859288cbc11f8f45e5c9c2349f0487b3bcb21\": container with ID starting with 0f027d9405f9db17345e77c83cb859288cbc11f8f45e5c9c2349f0487b3bcb21 not found: ID does not exist"
Oct 30 22:11:12.612269 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:12.612215    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"]
Oct 30 22:11:12.657811 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:12.657778    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-2f6bc1-openshift-default-7bdf5588c9-tcwfh"]
Oct 30 22:11:13.365454 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:13.365412    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df"]
Oct 30 22:11:13.366082 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:13.366064    4871 cpu_manager.go:414] "containerMap: removing container" podUID="42c86b8e-04b6-49fa-af33-9bc73297d162" containerName="istio-proxy"
Oct 30 22:11:13.366135 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:13.366084    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="42c86b8e-04b6-49fa-af33-9bc73297d162" containerName="istio-proxy"
Oct 30 22:11:13.366135 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:13.366103    4871 cpu_manager.go:414] "containerMap: removing container" podUID="07f3e008-0adf-4515-b983-fba8739d5ad9" containerName="maas-api"
Oct 30 22:11:13.366135 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:13.366108    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="07f3e008-0adf-4515-b983-fba8739d5ad9" containerName="maas-api"
Oct 30 22:11:13.366230 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:13.366193    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="42c86b8e-04b6-49fa-af33-9bc73297d162" containerName="istio-proxy"
Oct 30 22:11:13.366230 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:13.366205    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="07f3e008-0adf-4515-b983-fba8739d5ad9" containerName="maas-api"
Oct 30 22:11:13.366230 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:13.366211    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="42c86b8e-04b6-49fa-af33-9bc73297d162" containerName="istio-proxy"
Oct 30 22:11:13.371084 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:13.371064    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df"
Oct 30 22:11:13.425850 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:13.425818    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df"]
Oct 30 22:11:13.456480 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:13.454108    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41ad0caa-9e1c-4dc1-9f70-43beb36de4a7-maas-api-tls\") pod \"maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df\" (UID: \"41ad0caa-9e1c-4dc1-9f70-43beb36de4a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df"
Oct 30 22:11:13.456480 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:13.454162    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41ad0caa-9e1c-4dc1-9f70-43beb36de4a7-metrics-certs\") pod \"maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df\" (UID: \"41ad0caa-9e1c-4dc1-9f70-43beb36de4a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df"
Oct 30 22:11:13.456480 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:13.454300    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41ad0caa-9e1c-4dc1-9f70-43beb36de4a7-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df\" (UID: \"41ad0caa-9e1c-4dc1-9f70-43beb36de4a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df"
Oct 30 22:11:13.456480 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:13.454499    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzhjc\" (UniqueName: \"kubernetes.io/projected/41ad0caa-9e1c-4dc1-9f70-43beb36de4a7-kube-api-access-vzhjc\") pod \"maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df\" (UID: \"41ad0caa-9e1c-4dc1-9f70-43beb36de4a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df"
Oct 30 22:11:13.467703 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:13.467655    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-65659d-serving-cert\""
Oct 30 22:11:13.555705 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:13.555665    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41ad0caa-9e1c-4dc1-9f70-43beb36de4a7-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df\" (UID: \"41ad0caa-9e1c-4dc1-9f70-43beb36de4a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df"
Oct 30 22:11:13.555892 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:13.555794    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzhjc\" (UniqueName: \"kubernetes.io/projected/41ad0caa-9e1c-4dc1-9f70-43beb36de4a7-kube-api-access-vzhjc\") pod \"maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df\" (UID: \"41ad0caa-9e1c-4dc1-9f70-43beb36de4a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df"
Oct 30 22:11:13.555892 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:13.555822    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41ad0caa-9e1c-4dc1-9f70-43beb36de4a7-maas-api-tls\") pod \"maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df\" (UID: \"41ad0caa-9e1c-4dc1-9f70-43beb36de4a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df"
Oct 30 22:11:13.555892 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:13.555844    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41ad0caa-9e1c-4dc1-9f70-43beb36de4a7-metrics-certs\") pod \"maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df\" (UID: \"41ad0caa-9e1c-4dc1-9f70-43beb36de4a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df"
Oct 30 22:11:13.556075 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:11:13.555975    4871 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-65659d-serving-cert: secret "maas-api-e2e-ipp-a-65659d-serving-cert" not found
Oct 30 22:11:13.556075 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:11:13.556056    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41ad0caa-9e1c-4dc1-9f70-43beb36de4a7-maas-api-tls podName:41ad0caa-9e1c-4dc1-9f70-43beb36de4a7 nodeName:}" failed. No retries permitted until 2026-10-30 22:11:14.05603232 +0000 UTC m=+2016.233520932 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/41ad0caa-9e1c-4dc1-9f70-43beb36de4a7-maas-api-tls") pod "maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df" (UID: "41ad0caa-9e1c-4dc1-9f70-43beb36de4a7") : secret "maas-api-e2e-ipp-a-65659d-serving-cert" not found
Oct 30 22:11:13.556413 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:13.556390    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41ad0caa-9e1c-4dc1-9f70-43beb36de4a7-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df\" (UID: \"41ad0caa-9e1c-4dc1-9f70-43beb36de4a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df"
Oct 30 22:11:13.558229 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:13.558202    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41ad0caa-9e1c-4dc1-9f70-43beb36de4a7-metrics-certs\") pod \"maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df\" (UID: \"41ad0caa-9e1c-4dc1-9f70-43beb36de4a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df"
Oct 30 22:11:13.572690 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:13.572661    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:13.646306 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:13.643996    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzhjc\" (UniqueName: \"kubernetes.io/projected/41ad0caa-9e1c-4dc1-9f70-43beb36de4a7-kube-api-access-vzhjc\") pod \"maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df\" (UID: \"41ad0caa-9e1c-4dc1-9f70-43beb36de4a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df"
Oct 30 22:11:13.731634 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:13.731594    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:13.884167 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:13.884115    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:13.962491 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:13.962409    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="42c86b8e-04b6-49fa-af33-9bc73297d162" path="/var/lib/kubelet/pods/42c86b8e-04b6-49fa-af33-9bc73297d162/volumes"
Oct 30 22:11:14.077727 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:14.077687    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41ad0caa-9e1c-4dc1-9f70-43beb36de4a7-maas-api-tls\") pod \"maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df\" (UID: \"41ad0caa-9e1c-4dc1-9f70-43beb36de4a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df"
Oct 30 22:11:14.080126 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:14.080090    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41ad0caa-9e1c-4dc1-9f70-43beb36de4a7-maas-api-tls\") pod \"maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df\" (UID: \"41ad0caa-9e1c-4dc1-9f70-43beb36de4a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df"
Oct 30 22:11:14.285109 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:14.285022    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df"
Oct 30 22:11:14.669444 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:14.669414    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df"]
Oct 30 22:11:14.670080 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:11:14.670050    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41ad0caa_9e1c_4dc1_9f70_43beb36de4a7.slice/crio-a0c2837107c4dd6c0e3b9b3068b2dbc13471fd6a2b0ccf82bc758695c355a32e WatchSource:0}: Error finding container a0c2837107c4dd6c0e3b9b3068b2dbc13471fd6a2b0ccf82bc758695c355a32e: Status 404 returned error can't find the container with id a0c2837107c4dd6c0e3b9b3068b2dbc13471fd6a2b0ccf82bc758695c355a32e
Oct 30 22:11:15.483853 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:15.483814    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df" event={"ID":"41ad0caa-9e1c-4dc1-9f70-43beb36de4a7","Type":"ContainerStarted","Data":"a0c2837107c4dd6c0e3b9b3068b2dbc13471fd6a2b0ccf82bc758695c355a32e"}
Oct 30 22:11:16.492662 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:16.492623    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df" event={"ID":"41ad0caa-9e1c-4dc1-9f70-43beb36de4a7","Type":"ContainerStarted","Data":"7024a37a67b8de382f51d56ec9a5df0dd3f2b762d6c0a2a67c8c49bd1729c9e0"}
Oct 30 22:11:16.493169 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:16.492735    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df"
Oct 30 22:11:16.611281 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:16.611190    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df" podStartSLOduration=2.705189516 podStartE2EDuration="3.611170951s" totalImagesPullingTime="905.981435ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:11:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:11:16.604556387 +0000 UTC m=+2018.782045005" watchObservedRunningTime="2026-10-30 22:11:16.611170951 +0000 UTC m=+2018.788659579"
Oct 30 22:11:18.319898 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:18.319757    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:20.300428 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:20.300349    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:21.043258 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:21.042973    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"]
Oct 30 22:11:21.043654 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:21.043635    4871 cpu_manager.go:414] "containerMap: removing container" podUID="42c86b8e-04b6-49fa-af33-9bc73297d162" containerName="istio-proxy"
Oct 30 22:11:21.043729 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:21.043654    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="42c86b8e-04b6-49fa-af33-9bc73297d162" containerName="istio-proxy"
Oct 30 22:11:21.047779 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:21.047755    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"
Oct 30 22:11:21.096912 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:21.096864    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9b60b37-4047-499d-87fa-2533d0e12e85-workload-certs\") pod \"e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7\" (UID: \"f9b60b37-4047-499d-87fa-2533d0e12e85\") " pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"
Oct 30 22:11:21.097105 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:21.096950    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75hhs\" (UniqueName: \"kubernetes.io/projected/f9b60b37-4047-499d-87fa-2533d0e12e85-kube-api-access-75hhs\") pod \"e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7\" (UID: \"f9b60b37-4047-499d-87fa-2533d0e12e85\") " pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"
Oct 30 22:11:21.097105 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:21.097035    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9b60b37-4047-499d-87fa-2533d0e12e85-istio-podinfo\") pod \"e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7\" (UID: \"f9b60b37-4047-499d-87fa-2533d0e12e85\") " pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"
Oct 30 22:11:21.097228 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:21.097105    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9b60b37-4047-499d-87fa-2533d0e12e85-istiod-ca-cert\") pod \"e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7\" (UID: \"f9b60b37-4047-499d-87fa-2533d0e12e85\") " pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"
Oct 30 22:11:21.097228 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:21.097163    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9b60b37-4047-499d-87fa-2533d0e12e85-istio-token\") pod \"e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7\" (UID: \"f9b60b37-4047-499d-87fa-2533d0e12e85\") " pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"
Oct 30 22:11:21.097228 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:21.097201    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9b60b37-4047-499d-87fa-2533d0e12e85-istio-data\") pod \"e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7\" (UID: \"f9b60b37-4047-499d-87fa-2533d0e12e85\") " pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"
Oct 30 22:11:21.097419 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:21.097311    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9b60b37-4047-499d-87fa-2533d0e12e85-credential-socket\") pod \"e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7\" (UID: \"f9b60b37-4047-499d-87fa-2533d0e12e85\") " pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"
Oct 30 22:11:21.097419 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:21.097356    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9b60b37-4047-499d-87fa-2533d0e12e85-workload-socket\") pod \"e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7\" (UID: \"f9b60b37-4047-499d-87fa-2533d0e12e85\") " pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"
Oct 30 22:11:21.097535 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:21.097491    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9b60b37-4047-499d-87fa-2533d0e12e85-istio-envoy\") pod \"e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7\" (UID: \"f9b60b37-4047-499d-87fa-2533d0e12e85\") " pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"
Oct 30 22:11:21.098042 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:21.098020    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-a4257d-openshift-default-dockercfg-jk5v2\""
Oct 30 22:11:21.149311 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:21.149269    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"]
Oct 30 22:11:21.170388 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:21.170351    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:21.198800 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:21.198754    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9b60b37-4047-499d-87fa-2533d0e12e85-istio-envoy\") pod \"e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7\" (UID: \"f9b60b37-4047-499d-87fa-2533d0e12e85\") " pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"
Oct 30 22:11:21.199004 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:21.198845    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9b60b37-4047-499d-87fa-2533d0e12e85-workload-certs\") pod \"e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7\" (UID: \"f9b60b37-4047-499d-87fa-2533d0e12e85\") " pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"
Oct 30 22:11:21.199004 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:21.198889    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75hhs\" (UniqueName: \"kubernetes.io/projected/f9b60b37-4047-499d-87fa-2533d0e12e85-kube-api-access-75hhs\") pod \"e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7\" (UID: \"f9b60b37-4047-499d-87fa-2533d0e12e85\") " pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"
Oct 30 22:11:21.199004 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:21.198948    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9b60b37-4047-499d-87fa-2533d0e12e85-istio-podinfo\") pod \"e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7\" (UID: \"f9b60b37-4047-499d-87fa-2533d0e12e85\") " pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"
Oct 30 22:11:21.199004 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:21.198981    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9b60b37-4047-499d-87fa-2533d0e12e85-istiod-ca-cert\") pod \"e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7\" (UID: \"f9b60b37-4047-499d-87fa-2533d0e12e85\") " pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"
Oct 30 22:11:21.199256 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:21.199021    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9b60b37-4047-499d-87fa-2533d0e12e85-istio-token\") pod \"e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7\" (UID: \"f9b60b37-4047-499d-87fa-2533d0e12e85\") " pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"
Oct 30 22:11:21.199256 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:21.199057    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9b60b37-4047-499d-87fa-2533d0e12e85-istio-data\") pod \"e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7\" (UID: \"f9b60b37-4047-499d-87fa-2533d0e12e85\") " pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"
Oct 30 22:11:21.199256 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:21.199142    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9b60b37-4047-499d-87fa-2533d0e12e85-credential-socket\") pod \"e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7\" (UID: \"f9b60b37-4047-499d-87fa-2533d0e12e85\") " pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"
Oct 30 22:11:21.199256 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:21.199173    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9b60b37-4047-499d-87fa-2533d0e12e85-workload-socket\") pod \"e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7\" (UID: \"f9b60b37-4047-499d-87fa-2533d0e12e85\") " pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"
Oct 30 22:11:21.199598 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:21.199574    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9b60b37-4047-499d-87fa-2533d0e12e85-workload-socket\") pod \"e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7\" (UID: \"f9b60b37-4047-499d-87fa-2533d0e12e85\") " pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"
Oct 30 22:11:21.200160 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:21.200139    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9b60b37-4047-499d-87fa-2533d0e12e85-istio-data\") pod \"e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7\" (UID: \"f9b60b37-4047-499d-87fa-2533d0e12e85\") " pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"
Oct 30 22:11:21.200207 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:21.200142    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9b60b37-4047-499d-87fa-2533d0e12e85-istiod-ca-cert\") pod \"e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7\" (UID: \"f9b60b37-4047-499d-87fa-2533d0e12e85\") " pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"
Oct 30 22:11:21.200408 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:21.200387    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9b60b37-4047-499d-87fa-2533d0e12e85-credential-socket\") pod \"e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7\" (UID: \"f9b60b37-4047-499d-87fa-2533d0e12e85\") " pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"
Oct 30 22:11:21.200450 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:21.200429    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9b60b37-4047-499d-87fa-2533d0e12e85-workload-certs\") pod \"e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7\" (UID: \"f9b60b37-4047-499d-87fa-2533d0e12e85\") " pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"
Oct 30 22:11:21.201646 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:21.201620    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9b60b37-4047-499d-87fa-2533d0e12e85-istio-envoy\") pod \"e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7\" (UID: \"f9b60b37-4047-499d-87fa-2533d0e12e85\") " pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"
Oct 30 22:11:21.201802 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:21.201785    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9b60b37-4047-499d-87fa-2533d0e12e85-istio-podinfo\") pod \"e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7\" (UID: \"f9b60b37-4047-499d-87fa-2533d0e12e85\") " pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"
Oct 30 22:11:21.263498 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:21.263446    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75hhs\" (UniqueName: \"kubernetes.io/projected/f9b60b37-4047-499d-87fa-2533d0e12e85-kube-api-access-75hhs\") pod \"e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7\" (UID: \"f9b60b37-4047-499d-87fa-2533d0e12e85\") " pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"
Oct 30 22:11:21.264278 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:21.264255    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9b60b37-4047-499d-87fa-2533d0e12e85-istio-token\") pod \"e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7\" (UID: \"f9b60b37-4047-499d-87fa-2533d0e12e85\") " pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"
Oct 30 22:11:21.380058 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:21.374693    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:21.380058 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:21.378728    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"
Oct 30 22:11:21.859407 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:11:21.859370    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9b60b37_4047_499d_87fa_2533d0e12e85.slice/crio-45f4ccfc50436e283cc8d8f1338079fff2b425d1dabadc5087e0a80972040c8e WatchSource:0}: Error finding container 45f4ccfc50436e283cc8d8f1338079fff2b425d1dabadc5087e0a80972040c8e: Status 404 returned error can't find the container with id 45f4ccfc50436e283cc8d8f1338079fff2b425d1dabadc5087e0a80972040c8e
Oct 30 22:11:21.859698 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:21.859672    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"]
Oct 30 22:11:21.862088 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:21.862058    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:11:21.862149 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:21.862126    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:11:21.862188 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:21.862155    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:11:22.538266 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:22.537442    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df"
Oct 30 22:11:22.572895 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:22.572850    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7" event={"ID":"f9b60b37-4047-499d-87fa-2533d0e12e85","Type":"ContainerStarted","Data":"15224f9ad56a3e7cb9d01a90562ebfa96c8a99303668126beafec0be51cd51e0"}
Oct 30 22:11:22.573083 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:22.572918    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7" event={"ID":"f9b60b37-4047-499d-87fa-2533d0e12e85","Type":"ContainerStarted","Data":"45f4ccfc50436e283cc8d8f1338079fff2b425d1dabadc5087e0a80972040c8e"}
Oct 30 22:11:22.772734 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:22.771847    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7" podStartSLOduration=2.7718267599999997 podStartE2EDuration="2.77182676s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:11:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:11:22.768665328 +0000 UTC m=+2024.946153956" watchObservedRunningTime="2026-10-30 22:11:22.77182676 +0000 UTC m=+2024.949315379"
Oct 30 22:11:23.162541 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:23.162503    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:23.380454 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:23.380426    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"
Oct 30 22:11:23.387336 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:23.387301    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"
Oct 30 22:11:23.581908 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:23.581824    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"
Oct 30 22:11:23.582885 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:23.582863    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"
Oct 30 22:11:25.445817 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:25.445777    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:25.623123 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:25.623083    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:25.780181 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:25.780102    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:27.428356 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:27.428258    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:27.994316 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:27.994274    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"]
Oct 30 22:11:28.007631 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:28.007598    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"
Oct 30 22:11:28.065449 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:28.065413    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"]
Oct 30 22:11:28.093919 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:28.093879    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:28.104643 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:28.104609    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-95b84c-openshift-default-dockercfg-k22qk\""
Oct 30 22:11:28.134058 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:28.133342    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73ac22d3-ce8c-4c0c-a742-11d19526d964-istiod-ca-cert\") pod \"e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w\" (UID: \"73ac22d3-ce8c-4c0c-a742-11d19526d964\") " pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"
Oct 30 22:11:28.134058 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:28.133509    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73ac22d3-ce8c-4c0c-a742-11d19526d964-workload-certs\") pod \"e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w\" (UID: \"73ac22d3-ce8c-4c0c-a742-11d19526d964\") " pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"
Oct 30 22:11:28.134058 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:28.133546    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73ac22d3-ce8c-4c0c-a742-11d19526d964-istio-token\") pod \"e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w\" (UID: \"73ac22d3-ce8c-4c0c-a742-11d19526d964\") " pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"
Oct 30 22:11:28.134058 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:28.133577    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvgq9\" (UniqueName: \"kubernetes.io/projected/73ac22d3-ce8c-4c0c-a742-11d19526d964-kube-api-access-nvgq9\") pod \"e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w\" (UID: \"73ac22d3-ce8c-4c0c-a742-11d19526d964\") " pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"
Oct 30 22:11:28.134058 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:28.133618    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73ac22d3-ce8c-4c0c-a742-11d19526d964-istio-podinfo\") pod \"e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w\" (UID: \"73ac22d3-ce8c-4c0c-a742-11d19526d964\") " pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"
Oct 30 22:11:28.134058 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:28.133650    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73ac22d3-ce8c-4c0c-a742-11d19526d964-istio-data\") pod \"e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w\" (UID: \"73ac22d3-ce8c-4c0c-a742-11d19526d964\") " pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"
Oct 30 22:11:28.134058 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:28.133696    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73ac22d3-ce8c-4c0c-a742-11d19526d964-workload-socket\") pod \"e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w\" (UID: \"73ac22d3-ce8c-4c0c-a742-11d19526d964\") " pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"
Oct 30 22:11:28.134058 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:28.133792    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73ac22d3-ce8c-4c0c-a742-11d19526d964-credential-socket\") pod \"e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w\" (UID: \"73ac22d3-ce8c-4c0c-a742-11d19526d964\") " pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"
Oct 30 22:11:28.134058 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:28.133823    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73ac22d3-ce8c-4c0c-a742-11d19526d964-istio-envoy\") pod \"e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w\" (UID: \"73ac22d3-ce8c-4c0c-a742-11d19526d964\") " pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"
Oct 30 22:11:28.244264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:28.242017    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73ac22d3-ce8c-4c0c-a742-11d19526d964-istio-podinfo\") pod \"e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w\" (UID: \"73ac22d3-ce8c-4c0c-a742-11d19526d964\") " pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"
Oct 30 22:11:28.244264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:28.242079    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73ac22d3-ce8c-4c0c-a742-11d19526d964-istio-data\") pod \"e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w\" (UID: \"73ac22d3-ce8c-4c0c-a742-11d19526d964\") " pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"
Oct 30 22:11:28.244264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:28.242120    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73ac22d3-ce8c-4c0c-a742-11d19526d964-workload-socket\") pod \"e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w\" (UID: \"73ac22d3-ce8c-4c0c-a742-11d19526d964\") " pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"
Oct 30 22:11:28.244264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:28.242218    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73ac22d3-ce8c-4c0c-a742-11d19526d964-credential-socket\") pod \"e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w\" (UID: \"73ac22d3-ce8c-4c0c-a742-11d19526d964\") " pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"
Oct 30 22:11:28.244264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:28.242269    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73ac22d3-ce8c-4c0c-a742-11d19526d964-istio-envoy\") pod \"e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w\" (UID: \"73ac22d3-ce8c-4c0c-a742-11d19526d964\") " pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"
Oct 30 22:11:28.244264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:28.242399    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73ac22d3-ce8c-4c0c-a742-11d19526d964-istiod-ca-cert\") pod \"e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w\" (UID: \"73ac22d3-ce8c-4c0c-a742-11d19526d964\") " pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"
Oct 30 22:11:28.244264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:28.242478    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73ac22d3-ce8c-4c0c-a742-11d19526d964-workload-certs\") pod \"e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w\" (UID: \"73ac22d3-ce8c-4c0c-a742-11d19526d964\") " pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"
Oct 30 22:11:28.244264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:28.242502    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73ac22d3-ce8c-4c0c-a742-11d19526d964-istio-token\") pod \"e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w\" (UID: \"73ac22d3-ce8c-4c0c-a742-11d19526d964\") " pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"
Oct 30 22:11:28.244264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:28.242533    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvgq9\" (UniqueName: \"kubernetes.io/projected/73ac22d3-ce8c-4c0c-a742-11d19526d964-kube-api-access-nvgq9\") pod \"e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w\" (UID: \"73ac22d3-ce8c-4c0c-a742-11d19526d964\") " pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"
Oct 30 22:11:28.272306 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:28.271606    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73ac22d3-ce8c-4c0c-a742-11d19526d964-istiod-ca-cert\") pod \"e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w\" (UID: \"73ac22d3-ce8c-4c0c-a742-11d19526d964\") " pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"
Oct 30 22:11:28.272306 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:28.271879    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73ac22d3-ce8c-4c0c-a742-11d19526d964-workload-socket\") pod \"e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w\" (UID: \"73ac22d3-ce8c-4c0c-a742-11d19526d964\") " pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"
Oct 30 22:11:28.272306 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:28.272099    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73ac22d3-ce8c-4c0c-a742-11d19526d964-istio-data\") pod \"e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w\" (UID: \"73ac22d3-ce8c-4c0c-a742-11d19526d964\") " pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"
Oct 30 22:11:28.272658 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:28.272335    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73ac22d3-ce8c-4c0c-a742-11d19526d964-credential-socket\") pod \"e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w\" (UID: \"73ac22d3-ce8c-4c0c-a742-11d19526d964\") " pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"
Oct 30 22:11:28.274090 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:28.274065    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73ac22d3-ce8c-4c0c-a742-11d19526d964-workload-certs\") pod \"e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w\" (UID: \"73ac22d3-ce8c-4c0c-a742-11d19526d964\") " pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"
Oct 30 22:11:28.297657 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:28.297621    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73ac22d3-ce8c-4c0c-a742-11d19526d964-istio-envoy\") pod \"e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w\" (UID: \"73ac22d3-ce8c-4c0c-a742-11d19526d964\") " pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"
Oct 30 22:11:28.297863 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:28.297839    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73ac22d3-ce8c-4c0c-a742-11d19526d964-istio-podinfo\") pod \"e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w\" (UID: \"73ac22d3-ce8c-4c0c-a742-11d19526d964\") " pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"
Oct 30 22:11:28.323913 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:28.323877    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvgq9\" (UniqueName: \"kubernetes.io/projected/73ac22d3-ce8c-4c0c-a742-11d19526d964-kube-api-access-nvgq9\") pod \"e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w\" (UID: \"73ac22d3-ce8c-4c0c-a742-11d19526d964\") " pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"
Oct 30 22:11:28.341113 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:28.341043    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73ac22d3-ce8c-4c0c-a742-11d19526d964-istio-token\") pod \"e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w\" (UID: \"73ac22d3-ce8c-4c0c-a742-11d19526d964\") " pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"
Oct 30 22:11:28.398873 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:28.398835    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:28.559755 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:28.559653    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:28.639922 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:28.639875    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"
Oct 30 22:11:28.730868 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:28.730826    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:28.910080 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:28.903953    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:29.114506 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:29.109580    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"]
Oct 30 22:11:29.114506 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:11:29.112643    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73ac22d3_ce8c_4c0c_a742_11d19526d964.slice/crio-b22e22a57f8d41b9cdc23c32238cd391f5a3f8f3262f438c71e03378c38fa3b0 WatchSource:0}: Error finding container b22e22a57f8d41b9cdc23c32238cd391f5a3f8f3262f438c71e03378c38fa3b0: Status 404 returned error can't find the container with id b22e22a57f8d41b9cdc23c32238cd391f5a3f8f3262f438c71e03378c38fa3b0
Oct 30 22:11:29.116998 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:29.116961    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:11:29.117109 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:29.117035    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:11:29.117109 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:29.117072    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:11:29.654224 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:29.654177    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w" event={"ID":"73ac22d3-ce8c-4c0c-a742-11d19526d964","Type":"ContainerStarted","Data":"fd2b871d1f0d096fa2de6aa11147aa921d493260eabc795eb659d47b61da8ba5"}
Oct 30 22:11:29.655394 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:29.654257    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w" event={"ID":"73ac22d3-ce8c-4c0c-a742-11d19526d964","Type":"ContainerStarted","Data":"b22e22a57f8d41b9cdc23c32238cd391f5a3f8f3262f438c71e03378c38fa3b0"}
Oct 30 22:11:29.820851 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:29.820767    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w" podStartSLOduration=2.820753335 podStartE2EDuration="2.820753335s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:11:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:11:29.816883094 +0000 UTC m=+2031.994371763" watchObservedRunningTime="2026-10-30 22:11:29.820753335 +0000 UTC m=+2031.998241951"
Oct 30 22:11:30.643672 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:30.643624    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"
Oct 30 22:11:30.652604 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:30.652560    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"
Oct 30 22:11:30.663317 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:30.663278    4871 generic.go:356] "Generic (PLEG): container finished" podID="22d76305-7ecb-421f-9926-b1de7d91d2d8" containerID="5a883d0c34ef836dd4d3ec482f188166cbd34fdfaf1dbb97f50608ffff56f88d" exitCode=0
Oct 30 22:11:30.663704 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:30.663347    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj" event={"ID":"22d76305-7ecb-421f-9926-b1de7d91d2d8","Type":"ContainerDied","Data":"5a883d0c34ef836dd4d3ec482f188166cbd34fdfaf1dbb97f50608ffff56f88d"}
Oct 30 22:11:30.663797 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:30.663780    4871 scope.go:122] "RemoveContainer" containerID="5a883d0c34ef836dd4d3ec482f188166cbd34fdfaf1dbb97f50608ffff56f88d"
Oct 30 22:11:30.663884 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:30.663867    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"
Oct 30 22:11:30.664910 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:30.664883    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"
Oct 30 22:11:30.665392 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:30.665367    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:11:30.665452 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:30.665433    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:11:30.665497 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:30.665467    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:11:31.672644 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:31.672609    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj" event={"ID":"22d76305-7ecb-421f-9926-b1de7d91d2d8","Type":"ContainerStarted","Data":"7375cf92a0a97662af605be144db34dda8698b0ef9952014400b80cd21e7c659"}
Oct 30 22:11:32.100578 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:32.100488    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"
Oct 30 22:11:32.101769 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:32.101737    4871 patch_prober.go:29] interesting pod/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused" start-of-body=
Oct 30 22:11:32.101890 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:32.101796    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj" podUID="22d76305-7ecb-421f-9926-b1de7d91d2d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused"
Oct 30 22:11:32.566930 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:32.566880    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:32.774411 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:32.774364    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:32.960955 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:32.960914    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:33.100609 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:33.100573    4871 patch_prober.go:29] interesting pod/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused" start-of-body=
Oct 30 22:11:33.100801 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:33.100643    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj" podUID="22d76305-7ecb-421f-9926-b1de7d91d2d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused"
Oct 30 22:11:33.433263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:33.430195    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:34.099729 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:34.099692    4871 patch_prober.go:29] interesting pod/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused" start-of-body=
Oct 30 22:11:34.100135 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:34.099749    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj" podUID="22d76305-7ecb-421f-9926-b1de7d91d2d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused"
Oct 30 22:11:35.100631 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:35.100594    4871 patch_prober.go:29] interesting pod/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused" start-of-body=
Oct 30 22:11:35.101030 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:35.100651    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj" podUID="22d76305-7ecb-421f-9926-b1de7d91d2d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused"
Oct 30 22:11:35.569287 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:11:35.569256    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:11:36.100073 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:36.100035    4871 patch_prober.go:29] interesting pod/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused" start-of-body=
Oct 30 22:11:36.100326 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:36.100103    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj" podUID="22d76305-7ecb-421f-9926-b1de7d91d2d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused"
Oct 30 22:11:36.164451 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:36.164407    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"
Oct 30 22:11:37.106811 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:37.106774    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"
Oct 30 22:11:37.107968 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:37.107942    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"
Oct 30 22:11:38.389845 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:38.389804    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:38.588260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:38.586655    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:38.831317 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:38.827788    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:40.207888 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:40.207849    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:40.376182 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:40.376145    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:43.007641 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:43.007598    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:48.034470 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:48.034412    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:52.956658 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:52.956615    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f0b8c9-79d9577847-4wpgv"]
Oct 30 22:11:53.073050 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:53.072981    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:53.252784 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:53.252628    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:53.410859 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:53.410815    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:56.042128 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:56.042091    4871 generic.go:356] "Generic (PLEG): container finished" podID="78387fab-963c-4f00-9fda-33a5a5ebaee7" containerID="dd37dcafa918dc0f028207aebe5fd1fd38fca109ef8ab1c431aa6ca3b78ed693" exitCode=0
Oct 30 22:11:56.042763 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:56.042164    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk" event={"ID":"78387fab-963c-4f00-9fda-33a5a5ebaee7","Type":"ContainerDied","Data":"dd37dcafa918dc0f028207aebe5fd1fd38fca109ef8ab1c431aa6ca3b78ed693"}
Oct 30 22:11:56.042763 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:56.042688    4871 scope.go:122] "RemoveContainer" containerID="dd37dcafa918dc0f028207aebe5fd1fd38fca109ef8ab1c431aa6ca3b78ed693"
Oct 30 22:11:56.044263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:56.044212    4871 generic.go:356] "Generic (PLEG): container finished" podID="96df7172-8d94-4726-997e-2b0c9aeb8f5d" containerID="7d568e31411f948f4cf9680acd726967cd4fb7ebbcb98db95476ec161f5e201c" exitCode=0
Oct 30 22:11:56.044393 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:56.044305    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7" event={"ID":"96df7172-8d94-4726-997e-2b0c9aeb8f5d","Type":"ContainerDied","Data":"7d568e31411f948f4cf9680acd726967cd4fb7ebbcb98db95476ec161f5e201c"}
Oct 30 22:11:56.044487 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:56.044458    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:11:56.044564 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:56.044544    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:11:56.044642 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:56.044626    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:11:56.044751 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:56.044735    4871 scope.go:122] "RemoveContainer" containerID="7d568e31411f948f4cf9680acd726967cd4fb7ebbcb98db95476ec161f5e201c"
Oct 30 22:11:56.046298 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:56.046266    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:11:56.046387 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:56.046352    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:11:56.046433 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:56.046398    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:11:57.065580 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:57.065543    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk" event={"ID":"78387fab-963c-4f00-9fda-33a5a5ebaee7","Type":"ContainerStarted","Data":"2cf5840cc20a141c7287efd9c4f75eaa7fd4ee5b9022ffc78dabb19676590bc5"}
Oct 30 22:11:57.067819 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:57.067786    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7" event={"ID":"96df7172-8d94-4726-997e-2b0c9aeb8f5d","Type":"ContainerStarted","Data":"18a1b8d2852d239fe6b9af99f128bd5c89bee4de0808f52bac955f5f9bea7a63"}
Oct 30 22:11:57.345612 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:57.344964    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"
Oct 30 22:11:57.347392 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:57.347353    4871 patch_prober.go:29] interesting pod/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused" start-of-body=
Oct 30 22:11:57.347638 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:57.347612    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7" podUID="96df7172-8d94-4726-997e-2b0c9aeb8f5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused"
Oct 30 22:11:57.591986 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:57.591946    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"
Oct 30 22:11:57.593330 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:57.593304    4871 patch_prober.go:29] interesting pod/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.134:15021/healthz/ready\": dial tcp 10.133.0.134:15021: connect: connection refused" start-of-body=
Oct 30 22:11:57.593411 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:57.593363    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk" podUID="78387fab-963c-4f00-9fda-33a5a5ebaee7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.134:15021/healthz/ready\": dial tcp 10.133.0.134:15021: connect: connection refused"
Oct 30 22:11:57.957261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:57.956929    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f0b8c9-79d9577847-4wpgv" podUID="661be396-da87-40b1-bfab-7736db05daf0" containerName="payload-pre-processing" containerID="cri-o://d8d11a3a1ed8c2ab24567cce37ca565746b1c73a0929cacc40e8e16e5aa8f9ca" gracePeriod=25
Oct 30 22:11:58.053535 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:58.053487    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"
Oct 30 22:11:58.079468 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:58.079432    4871 generic.go:356] "Generic (PLEG): container finished" podID="661be396-da87-40b1-bfab-7736db05daf0" containerID="d8d11a3a1ed8c2ab24567cce37ca565746b1c73a0929cacc40e8e16e5aa8f9ca" exitCode=0
Oct 30 22:11:58.079799 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:58.079505    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f0b8c9-79d9577847-4wpgv" event={"ID":"661be396-da87-40b1-bfab-7736db05daf0","Type":"ContainerDied","Data":"d8d11a3a1ed8c2ab24567cce37ca565746b1c73a0929cacc40e8e16e5aa8f9ca"}
Oct 30 22:11:58.143008 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:58.142924    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:58.358800 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:58.358708    4871 patch_prober.go:29] interesting pod/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused" start-of-body=
Oct 30 22:11:58.358800 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:58.358779    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7" podUID="96df7172-8d94-4726-997e-2b0c9aeb8f5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused"
Oct 30 22:11:58.465263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:58.464409    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f0b8c9-79d9577847-4wpgv"
Oct 30 22:11:58.575686 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:58.575568    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/661be396-da87-40b1-bfab-7736db05daf0-kube-api-access-dnswd\" (UniqueName: \"kubernetes.io/projected/661be396-da87-40b1-bfab-7736db05daf0-kube-api-access-dnswd\") pod \"661be396-da87-40b1-bfab-7736db05daf0\" (UID: \"661be396-da87-40b1-bfab-7736db05daf0\") "
Oct 30 22:11:58.575686 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:58.575655    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/661be396-da87-40b1-bfab-7736db05daf0-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/661be396-da87-40b1-bfab-7736db05daf0-plugins-config-volume\") pod \"661be396-da87-40b1-bfab-7736db05daf0\" (UID: \"661be396-da87-40b1-bfab-7736db05daf0\") "
Oct 30 22:11:58.576443 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:58.576412    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/661be396-da87-40b1-bfab-7736db05daf0-plugins-config-volume" pod "661be396-da87-40b1-bfab-7736db05daf0" (UID: "661be396-da87-40b1-bfab-7736db05daf0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:11:58.578434 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:58.578406    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/661be396-da87-40b1-bfab-7736db05daf0-kube-api-access-dnswd" pod "661be396-da87-40b1-bfab-7736db05daf0" (UID: "661be396-da87-40b1-bfab-7736db05daf0"). InnerVolumeSpecName "kube-api-access-dnswd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:58.593723 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:58.593551    4871 patch_prober.go:29] interesting pod/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.134:15021/healthz/ready\": dial tcp 10.133.0.134:15021: connect: connection refused" start-of-body=
Oct 30 22:11:58.593723 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:58.593632    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk" podUID="78387fab-963c-4f00-9fda-33a5a5ebaee7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.134:15021/healthz/ready\": dial tcp 10.133.0.134:15021: connect: connection refused"
Oct 30 22:11:58.677880 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:58.677831    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dnswd\" (UniqueName: \"kubernetes.io/projected/661be396-da87-40b1-bfab-7736db05daf0-kube-api-access-dnswd\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:11:58.677880 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:58.677878    4871 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/661be396-da87-40b1-bfab-7736db05daf0-plugins-config-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:11:59.090009 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:59.089976    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f0b8c9-79d9577847-4wpgv"
Oct 30 22:11:59.090009 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:59.089999    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f0b8c9-79d9577847-4wpgv" event={"ID":"661be396-da87-40b1-bfab-7736db05daf0","Type":"ContainerDied","Data":"798b60ce5edf4313338fe44d6f0e79dd0ad93a80df99bedba84d932ce30e0c0f"}
Oct 30 22:11:59.090500 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:59.090057    4871 scope.go:122] "RemoveContainer" containerID="d8d11a3a1ed8c2ab24567cce37ca565746b1c73a0929cacc40e8e16e5aa8f9ca"
Oct 30 22:11:59.258658 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:59.258619    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f0b8c9-79d9577847-4wpgv"]
Oct 30 22:11:59.262552 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:59.262513    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f0b8c9-79d9577847-4wpgv"]
Oct 30 22:11:59.347807 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:59.347703    4871 patch_prober.go:29] interesting pod/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused" start-of-body=
Oct 30 22:11:59.348101 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:59.348065    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7" podUID="96df7172-8d94-4726-997e-2b0c9aeb8f5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused"
Oct 30 22:11:59.554464 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:59.552477    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"
Oct 30 22:11:59.592007 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:59.591972    4871 patch_prober.go:29] interesting pod/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.134:15021/healthz/ready\": dial tcp 10.133.0.134:15021: connect: connection refused" start-of-body=
Oct 30 22:11:59.592187 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:59.592019    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk" podUID="78387fab-963c-4f00-9fda-33a5a5ebaee7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.134:15021/healthz/ready\": dial tcp 10.133.0.134:15021: connect: connection refused"
Oct 30 22:11:59.965831 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:11:59.965784    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="661be396-da87-40b1-bfab-7736db05daf0" path="/var/lib/kubelet/pods/661be396-da87-40b1-bfab-7736db05daf0/volumes"
Oct 30 22:12:00.344809 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:00.344702    4871 patch_prober.go:29] interesting pod/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused" start-of-body=
Oct 30 22:12:00.345463 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:00.345423    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7" podUID="96df7172-8d94-4726-997e-2b0c9aeb8f5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused"
Oct 30 22:12:00.592026 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:00.591990    4871 patch_prober.go:29] interesting pod/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.134:15021/healthz/ready\": dial tcp 10.133.0.134:15021: connect: connection refused" start-of-body=
Oct 30 22:12:00.592288 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:00.592048    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk" podUID="78387fab-963c-4f00-9fda-33a5a5ebaee7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.134:15021/healthz/ready\": dial tcp 10.133.0.134:15021: connect: connection refused"
Oct 30 22:12:01.344746 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:01.344708    4871 patch_prober.go:29] interesting pod/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused" start-of-body=
Oct 30 22:12:01.345030 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:01.345001    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7" podUID="96df7172-8d94-4726-997e-2b0c9aeb8f5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused"
Oct 30 22:12:01.592175 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:01.592128    4871 patch_prober.go:29] interesting pod/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.134:15021/healthz/ready\": dial tcp 10.133.0.134:15021: connect: connection refused" start-of-body=
Oct 30 22:12:01.592400 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:01.592197    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk" podUID="78387fab-963c-4f00-9fda-33a5a5ebaee7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.134:15021/healthz/ready\": dial tcp 10.133.0.134:15021: connect: connection refused"
Oct 30 22:12:02.343392 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:02.343352    4871 patch_prober.go:29] interesting pod/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused" start-of-body=
Oct 30 22:12:02.343629 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:02.343432    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7" podUID="96df7172-8d94-4726-997e-2b0c9aeb8f5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused"
Oct 30 22:12:02.597116 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:02.597035    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"
Oct 30 22:12:02.598196 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:02.598161    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"
Oct 30 22:12:03.116575 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:03.116524    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:03.347776 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:03.347745    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"
Oct 30 22:12:03.348935 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:03.348915    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"
Oct 30 22:12:05.570658 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:12:05.570606    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:12:07.722967 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:07.722928    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:12.612947 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:12.612904    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:12.798860 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:12.798821    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:12.950552 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:12.950456    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:13.148515 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:13.148440    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:16.981056 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:16.981014    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf"]
Oct 30 22:12:16.981834 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:16.981812    4871 cpu_manager.go:414] "containerMap: removing container" podUID="661be396-da87-40b1-bfab-7736db05daf0" containerName="payload-pre-processing"
Oct 30 22:12:16.981834 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:16.981833    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="661be396-da87-40b1-bfab-7736db05daf0" containerName="payload-pre-processing"
Oct 30 22:12:16.981948 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:16.981921    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="661be396-da87-40b1-bfab-7736db05daf0" containerName="payload-pre-processing"
Oct 30 22:12:16.984763 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:16.984740    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf"
Oct 30 22:12:17.082958 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:17.082930    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-a4257d-serving-cert\""
Oct 30 22:12:17.087842 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:17.087811    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf"]
Oct 30 22:12:17.092604 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:17.092575    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qwjx\" (UniqueName: \"kubernetes.io/projected/9d801bc6-50a2-4eeb-9038-f3758bf7bf6a-kube-api-access-4qwjx\") pod \"maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf\" (UID: \"9d801bc6-50a2-4eeb-9038-f3758bf7bf6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf"
Oct 30 22:12:17.092737 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:17.092717    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d801bc6-50a2-4eeb-9038-f3758bf7bf6a-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf\" (UID: \"9d801bc6-50a2-4eeb-9038-f3758bf7bf6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf"
Oct 30 22:12:17.092892 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:17.092870    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d801bc6-50a2-4eeb-9038-f3758bf7bf6a-metrics-certs\") pod \"maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf\" (UID: \"9d801bc6-50a2-4eeb-9038-f3758bf7bf6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf"
Oct 30 22:12:17.093083 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:17.093057    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9d801bc6-50a2-4eeb-9038-f3758bf7bf6a-maas-api-tls\") pod \"maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf\" (UID: \"9d801bc6-50a2-4eeb-9038-f3758bf7bf6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf"
Oct 30 22:12:17.194270 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:17.194205    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9d801bc6-50a2-4eeb-9038-f3758bf7bf6a-maas-api-tls\") pod \"maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf\" (UID: \"9d801bc6-50a2-4eeb-9038-f3758bf7bf6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf"
Oct 30 22:12:17.194463 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:17.194356    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qwjx\" (UniqueName: \"kubernetes.io/projected/9d801bc6-50a2-4eeb-9038-f3758bf7bf6a-kube-api-access-4qwjx\") pod \"maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf\" (UID: \"9d801bc6-50a2-4eeb-9038-f3758bf7bf6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf"
Oct 30 22:12:17.194463 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:17.194395    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d801bc6-50a2-4eeb-9038-f3758bf7bf6a-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf\" (UID: \"9d801bc6-50a2-4eeb-9038-f3758bf7bf6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf"
Oct 30 22:12:17.194543 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:17.194470    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d801bc6-50a2-4eeb-9038-f3758bf7bf6a-metrics-certs\") pod \"maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf\" (UID: \"9d801bc6-50a2-4eeb-9038-f3758bf7bf6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf"
Oct 30 22:12:17.195136 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:17.195109    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d801bc6-50a2-4eeb-9038-f3758bf7bf6a-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf\" (UID: \"9d801bc6-50a2-4eeb-9038-f3758bf7bf6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf"
Oct 30 22:12:17.196714 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:17.196681    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:17.197038 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:17.197018    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d801bc6-50a2-4eeb-9038-f3758bf7bf6a-metrics-certs\") pod \"maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf\" (UID: \"9d801bc6-50a2-4eeb-9038-f3758bf7bf6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf"
Oct 30 22:12:17.197161 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:17.197142    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9d801bc6-50a2-4eeb-9038-f3758bf7bf6a-maas-api-tls\") pod \"maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf\" (UID: \"9d801bc6-50a2-4eeb-9038-f3758bf7bf6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf"
Oct 30 22:12:17.252831 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:17.252753    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qwjx\" (UniqueName: \"kubernetes.io/projected/9d801bc6-50a2-4eeb-9038-f3758bf7bf6a-kube-api-access-4qwjx\") pod \"maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf\" (UID: \"9d801bc6-50a2-4eeb-9038-f3758bf7bf6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf"
Oct 30 22:12:17.298433 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:17.298388    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf"
Oct 30 22:12:17.375156 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:17.374778    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:17.548357 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:17.546784    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:17.675199 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:17.675166    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf"]
Oct 30 22:12:17.676511 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:12:17.676482    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d801bc6_50a2_4eeb_9038_f3758bf7bf6a.slice/crio-f9c2e13178c59a066aab9d64484451fe3e411ce3fe3b7556463ad08fa25297a9 WatchSource:0}: Error finding container f9c2e13178c59a066aab9d64484451fe3e411ce3fe3b7556463ad08fa25297a9: Status 404 returned error can't find the container with id f9c2e13178c59a066aab9d64484451fe3e411ce3fe3b7556463ad08fa25297a9
Oct 30 22:12:18.314919 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:18.314881    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf" event={"ID":"9d801bc6-50a2-4eeb-9038-f3758bf7bf6a","Type":"ContainerStarted","Data":"f9c2e13178c59a066aab9d64484451fe3e411ce3fe3b7556463ad08fa25297a9"}
Oct 30 22:12:19.323310 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:19.323267    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf" event={"ID":"9d801bc6-50a2-4eeb-9038-f3758bf7bf6a","Type":"ContainerStarted","Data":"87bf0ba967e77ec9520e5389c924d571dd5cd7dcace054cd72e08f72edd993a1"}
Oct 30 22:12:19.323718 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:19.323332    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf"
Oct 30 22:12:19.438982 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:19.438921    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf" podStartSLOduration=2.528649773 podStartE2EDuration="3.438905637s" totalImagesPullingTime="910.255864ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:12:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:12:19.438022765 +0000 UTC m=+2081.615511382" watchObservedRunningTime="2026-10-30 22:12:19.438905637 +0000 UTC m=+2081.616394254"
Oct 30 22:12:21.897894 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:21.897858    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:24.448919 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:24.448876    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:24.801688 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:24.801553    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:25.335418 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:25.335385    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf"
Oct 30 22:12:26.571543 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:26.571502    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:26.722542 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:26.722484    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:26.883336 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:26.881093    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:31.651043 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:31.651009    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:32.258501 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:32.258463    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:35.569612 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:12:35.569574    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:12:36.575965 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:36.575928    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:38.360286 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:38.360251    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx_8d401654-8a4c-4a79-b2d3-612765312421/extract/0.log"
Oct 30 22:12:38.364873 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:38.364850    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx_8d401654-8a4c-4a79-b2d3-612765312421/extract/0.log"
Oct 30 22:12:38.377069 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:38.377039    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/2.log"
Oct 30 22:12:38.380596 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:38.380569    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/2.log"
Oct 30 22:12:38.384572 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:38.384553    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzntd_2360fddc-52a5-42a5-82ca-a05aff84d217/ovn-acl-logging/0.log"
Oct 30 22:12:38.388145 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:38.388126    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzntd_2360fddc-52a5-42a5-82ca-a05aff84d217/ovn-acl-logging/0.log"
Oct 30 22:12:40.533407 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:40.533366    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:41.695837 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:41.695803    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:42.584086 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:42.583905    4871 generic.go:356] "Generic (PLEG): container finished" podID="f9b60b37-4047-499d-87fa-2533d0e12e85" containerID="15224f9ad56a3e7cb9d01a90562ebfa96c8a99303668126beafec0be51cd51e0" exitCode=0
Oct 30 22:12:42.584086 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:42.583956    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7" event={"ID":"f9b60b37-4047-499d-87fa-2533d0e12e85","Type":"ContainerDied","Data":"15224f9ad56a3e7cb9d01a90562ebfa96c8a99303668126beafec0be51cd51e0"}
Oct 30 22:12:42.584540 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:42.584519    4871 scope.go:122] "RemoveContainer" containerID="15224f9ad56a3e7cb9d01a90562ebfa96c8a99303668126beafec0be51cd51e0"
Oct 30 22:12:42.586557 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:42.586525    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:12:42.586665 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:42.586596    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:12:42.586665 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:42.586655    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:12:43.594563 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:43.594526    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7" event={"ID":"f9b60b37-4047-499d-87fa-2533d0e12e85","Type":"ContainerStarted","Data":"b28eac6dad69f9d340415115e0f584821e43a968cc88c5854c6d6a0bd41026bf"}
Oct 30 22:12:44.381093 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:44.381053    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"
Oct 30 22:12:44.381347 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:44.381322    4871 patch_prober.go:29] interesting pod/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.141:15021/healthz/ready\": dial tcp 10.133.0.141:15021: connect: connection refused" start-of-body=
Oct 30 22:12:44.381400 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:44.381374    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7" podUID="f9b60b37-4047-499d-87fa-2533d0e12e85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.141:15021/healthz/ready\": dial tcp 10.133.0.141:15021: connect: connection refused"
Oct 30 22:12:45.129303 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:45.129203    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:45.382730 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:45.382638    4871 patch_prober.go:29] interesting pod/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.141:15021/healthz/ready\": dial tcp 10.133.0.141:15021: connect: connection refused" start-of-body=
Oct 30 22:12:45.383271 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:45.383213    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7" podUID="f9b60b37-4047-499d-87fa-2533d0e12e85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.141:15021/healthz/ready\": dial tcp 10.133.0.141:15021: connect: connection refused"
Oct 30 22:12:45.599783 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:45.599733    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:45.876194 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:45.873871    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:46.029619 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:46.029581    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:46.381145 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:46.381108    4871 patch_prober.go:29] interesting pod/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.141:15021/healthz/ready\": dial tcp 10.133.0.141:15021: connect: connection refused" start-of-body=
Oct 30 22:12:46.381582 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:46.381164    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7" podUID="f9b60b37-4047-499d-87fa-2533d0e12e85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.141:15021/healthz/ready\": dial tcp 10.133.0.141:15021: connect: connection refused"
Oct 30 22:12:47.385265 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:47.381768    4871 patch_prober.go:29] interesting pod/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.141:15021/healthz/ready\": dial tcp 10.133.0.141:15021: connect: connection refused" start-of-body=
Oct 30 22:12:47.385265 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:47.381845    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7" podUID="f9b60b37-4047-499d-87fa-2533d0e12e85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.141:15021/healthz/ready\": dial tcp 10.133.0.141:15021: connect: connection refused"
Oct 30 22:12:47.733941 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:47.733840    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"]
Oct 30 22:12:47.734146 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:47.734115    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52" podUID="8937fa65-1fb5-4614-9be1-9075e2b3f46b" containerName="istio-proxy" containerID="cri-o://9c8cd1f4cefef255025a31f10f048eba8e711ea542dc7f633f3497981b512888" gracePeriod=30
Oct 30 22:12:48.337725 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:48.337692    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:48.390262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:48.388365    4871 patch_prober.go:29] interesting pod/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.141:15021/healthz/ready\": dial tcp 10.133.0.141:15021: connect: connection refused" start-of-body=
Oct 30 22:12:48.390262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:48.388431    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7" podUID="f9b60b37-4047-499d-87fa-2533d0e12e85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.141:15021/healthz/ready\": dial tcp 10.133.0.141:15021: connect: connection refused"
Oct 30 22:12:49.386374 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:49.386329    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"
Oct 30 22:12:49.386779 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:49.386747    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"
Oct 30 22:12:49.387583 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:49.387560    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"
Oct 30 22:12:51.764716 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:51.764683    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:12:51.765139 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:51.764747    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52" podUID="8937fa65-1fb5-4614-9be1-9075e2b3f46b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:12:53.004714 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:53.004663    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:53.071984 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:53.071952    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"
Oct 30 22:12:53.166411 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:53.166366    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8937fa65-1fb5-4614-9be1-9075e2b3f46b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8937fa65-1fb5-4614-9be1-9075e2b3f46b-istio-podinfo\") pod \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\" (UID: \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\") "
Oct 30 22:12:53.166622 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:53.166470    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8937fa65-1fb5-4614-9be1-9075e2b3f46b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8937fa65-1fb5-4614-9be1-9075e2b3f46b-credential-socket\") pod \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\" (UID: \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\") "
Oct 30 22:12:53.166622 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:53.166521    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8937fa65-1fb5-4614-9be1-9075e2b3f46b-istio-token\" (UniqueName: \"kubernetes.io/projected/8937fa65-1fb5-4614-9be1-9075e2b3f46b-istio-token\") pod \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\" (UID: \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\") "
Oct 30 22:12:53.166752 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:53.166637    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8937fa65-1fb5-4614-9be1-9075e2b3f46b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8937fa65-1fb5-4614-9be1-9075e2b3f46b-istiod-ca-cert\") pod \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\" (UID: \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\") "
Oct 30 22:12:53.166752 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:53.166687    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8937fa65-1fb5-4614-9be1-9075e2b3f46b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8937fa65-1fb5-4614-9be1-9075e2b3f46b-istio-data\") pod \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\" (UID: \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\") "
Oct 30 22:12:53.166752 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:53.166727    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8937fa65-1fb5-4614-9be1-9075e2b3f46b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8937fa65-1fb5-4614-9be1-9075e2b3f46b-workload-certs\") pod \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\" (UID: \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\") "
Oct 30 22:12:53.166922 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:53.166785    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8937fa65-1fb5-4614-9be1-9075e2b3f46b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8937fa65-1fb5-4614-9be1-9075e2b3f46b-istio-envoy\") pod \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\" (UID: \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\") "
Oct 30 22:12:53.166922 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:53.166829    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8937fa65-1fb5-4614-9be1-9075e2b3f46b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8937fa65-1fb5-4614-9be1-9075e2b3f46b-workload-socket\") pod \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\" (UID: \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\") "
Oct 30 22:12:53.166922 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:53.166875    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8937fa65-1fb5-4614-9be1-9075e2b3f46b-kube-api-access-kbjfh\" (UniqueName: \"kubernetes.io/projected/8937fa65-1fb5-4614-9be1-9075e2b3f46b-kube-api-access-kbjfh\") pod \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\" (UID: \"8937fa65-1fb5-4614-9be1-9075e2b3f46b\") "
Oct 30 22:12:53.167086 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:53.167065    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8937fa65-1fb5-4614-9be1-9075e2b3f46b-workload-certs" pod "8937fa65-1fb5-4614-9be1-9075e2b3f46b" (UID: "8937fa65-1fb5-4614-9be1-9075e2b3f46b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:12:53.167151 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:53.167101    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8937fa65-1fb5-4614-9be1-9075e2b3f46b-credential-socket" pod "8937fa65-1fb5-4614-9be1-9075e2b3f46b" (UID: "8937fa65-1fb5-4614-9be1-9075e2b3f46b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:12:53.167151 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:53.167088    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8937fa65-1fb5-4614-9be1-9075e2b3f46b-istio-data" pod "8937fa65-1fb5-4614-9be1-9075e2b3f46b" (UID: "8937fa65-1fb5-4614-9be1-9075e2b3f46b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:12:53.167594 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:53.167381    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8937fa65-1fb5-4614-9be1-9075e2b3f46b-workload-socket" pod "8937fa65-1fb5-4614-9be1-9075e2b3f46b" (UID: "8937fa65-1fb5-4614-9be1-9075e2b3f46b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:12:53.167594 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:53.167516    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8937fa65-1fb5-4614-9be1-9075e2b3f46b-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:12:53.167594 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:53.167539    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8937fa65-1fb5-4614-9be1-9075e2b3f46b-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:12:53.167594 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:53.167554    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8937fa65-1fb5-4614-9be1-9075e2b3f46b-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:12:53.167594 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:53.167567    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8937fa65-1fb5-4614-9be1-9075e2b3f46b-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:12:53.168469 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:53.168423    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8937fa65-1fb5-4614-9be1-9075e2b3f46b-istiod-ca-cert" pod "8937fa65-1fb5-4614-9be1-9075e2b3f46b" (UID: "8937fa65-1fb5-4614-9be1-9075e2b3f46b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:12:53.169777 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:53.169750    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8937fa65-1fb5-4614-9be1-9075e2b3f46b-istio-token" pod "8937fa65-1fb5-4614-9be1-9075e2b3f46b" (UID: "8937fa65-1fb5-4614-9be1-9075e2b3f46b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:12:53.170060 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:53.170027    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8937fa65-1fb5-4614-9be1-9075e2b3f46b-istio-envoy" pod "8937fa65-1fb5-4614-9be1-9075e2b3f46b" (UID: "8937fa65-1fb5-4614-9be1-9075e2b3f46b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:12:53.170468 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:53.170445    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8937fa65-1fb5-4614-9be1-9075e2b3f46b-kube-api-access-kbjfh" pod "8937fa65-1fb5-4614-9be1-9075e2b3f46b" (UID: "8937fa65-1fb5-4614-9be1-9075e2b3f46b"). InnerVolumeSpecName "kube-api-access-kbjfh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:12:53.170739 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:53.170720    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8937fa65-1fb5-4614-9be1-9075e2b3f46b-istio-podinfo" pod "8937fa65-1fb5-4614-9be1-9075e2b3f46b" (UID: "8937fa65-1fb5-4614-9be1-9075e2b3f46b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:12:53.268943 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:53.268892    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8937fa65-1fb5-4614-9be1-9075e2b3f46b-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:12:53.268943 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:53.268928    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8937fa65-1fb5-4614-9be1-9075e2b3f46b-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:12:53.268943 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:53.268939    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8937fa65-1fb5-4614-9be1-9075e2b3f46b-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:12:53.268943 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:53.268949    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbjfh\" (UniqueName: \"kubernetes.io/projected/8937fa65-1fb5-4614-9be1-9075e2b3f46b-kube-api-access-kbjfh\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:12:53.269347 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:53.268961    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8937fa65-1fb5-4614-9be1-9075e2b3f46b-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:12:53.732254 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:53.732207    4871 generic.go:356] "Generic (PLEG): container finished" podID="8937fa65-1fb5-4614-9be1-9075e2b3f46b" containerID="9c8cd1f4cefef255025a31f10f048eba8e711ea542dc7f633f3497981b512888" exitCode=0
Oct 30 22:12:53.732459 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:53.732271    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52" event={"ID":"8937fa65-1fb5-4614-9be1-9075e2b3f46b","Type":"ContainerDied","Data":"9c8cd1f4cefef255025a31f10f048eba8e711ea542dc7f633f3497981b512888"}
Oct 30 22:12:53.732459 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:53.732294    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"
Oct 30 22:12:53.732459 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:53.732331    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52" event={"ID":"8937fa65-1fb5-4614-9be1-9075e2b3f46b","Type":"ContainerDied","Data":"e08e7b9d37e6a8a5c164a13ef3c4e55cde40efdeb7c03c22f93f01e9b139ff41"}
Oct 30 22:12:53.732459 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:53.732358    4871 scope.go:122] "RemoveContainer" containerID="9c8cd1f4cefef255025a31f10f048eba8e711ea542dc7f633f3497981b512888"
Oct 30 22:12:53.744755 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:53.744730    4871 scope.go:122] "RemoveContainer" containerID="9c8cd1f4cefef255025a31f10f048eba8e711ea542dc7f633f3497981b512888"
Oct 30 22:12:53.745278 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:12:53.745227    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c8cd1f4cefef255025a31f10f048eba8e711ea542dc7f633f3497981b512888\": container with ID starting with 9c8cd1f4cefef255025a31f10f048eba8e711ea542dc7f633f3497981b512888 not found: ID does not exist" containerID="9c8cd1f4cefef255025a31f10f048eba8e711ea542dc7f633f3497981b512888"
Oct 30 22:12:53.745377 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:53.745289    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c8cd1f4cefef255025a31f10f048eba8e711ea542dc7f633f3497981b512888"} err="failed to get container status \"9c8cd1f4cefef255025a31f10f048eba8e711ea542dc7f633f3497981b512888\": rpc error: code = NotFound desc = could not find container \"9c8cd1f4cefef255025a31f10f048eba8e711ea542dc7f633f3497981b512888\": container with ID starting with 9c8cd1f4cefef255025a31f10f048eba8e711ea542dc7f633f3497981b512888 not found: ID does not exist"
Oct 30 22:12:53.891166 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:53.891129    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"]
Oct 30 22:12:53.938272 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:53.938225    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-vhn52"]
Oct 30 22:12:53.963045 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:53.963010    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8937fa65-1fb5-4614-9be1-9075e2b3f46b" path="/var/lib/kubelet/pods/8937fa65-1fb5-4614-9be1-9075e2b3f46b/volumes"
Oct 30 22:12:57.777633 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:12:57.777602    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:01.197517 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:01.197466    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:01.742068 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:01.742034    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:04.335018 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:04.334982    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:05.571536 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:13:05.571496    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:13:08.908556 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:08.908489    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:09.315602 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:09.315494    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:09.473964 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:09.473915    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:10.904256 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:10.904201    4871 generic.go:356] "Generic (PLEG): container finished" podID="73ac22d3-ce8c-4c0c-a742-11d19526d964" containerID="fd2b871d1f0d096fa2de6aa11147aa921d493260eabc795eb659d47b61da8ba5" exitCode=0
Oct 30 22:13:10.904809 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:10.904289    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w" event={"ID":"73ac22d3-ce8c-4c0c-a742-11d19526d964","Type":"ContainerDied","Data":"fd2b871d1f0d096fa2de6aa11147aa921d493260eabc795eb659d47b61da8ba5"}
Oct 30 22:13:10.904809 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:10.904791    4871 scope.go:122] "RemoveContainer" containerID="fd2b871d1f0d096fa2de6aa11147aa921d493260eabc795eb659d47b61da8ba5"
Oct 30 22:13:10.906397 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:10.906365    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:13:10.906529 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:10.906440    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:13:10.906529 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:10.906473    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:13:11.941589 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:11.941550    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w" event={"ID":"73ac22d3-ce8c-4c0c-a742-11d19526d964","Type":"ContainerStarted","Data":"e786a485f0980baa83e4d1e469cf0a56ff9f458b69756e568ccecd3254e8ad50"}
Oct 30 22:13:12.640654 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:12.640596    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"
Oct 30 22:13:12.641885 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:12.641850    4871 patch_prober.go:29] interesting pod/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.142:15021/healthz/ready\": dial tcp 10.133.0.142:15021: connect: connection refused" start-of-body=
Oct 30 22:13:12.642052 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:12.641912    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w" podUID="73ac22d3-ce8c-4c0c-a742-11d19526d964" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.142:15021/healthz/ready\": dial tcp 10.133.0.142:15021: connect: connection refused"
Oct 30 22:13:13.560108 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:13.560072    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:13.645809 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:13.645766    4871 patch_prober.go:29] interesting pod/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.142:15021/healthz/ready\": dial tcp 10.133.0.142:15021: connect: connection refused" start-of-body=
Oct 30 22:13:13.646201 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:13.646070    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w" podUID="73ac22d3-ce8c-4c0c-a742-11d19526d964" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.142:15021/healthz/ready\": dial tcp 10.133.0.142:15021: connect: connection refused"
Oct 30 22:13:14.641115 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:14.641078    4871 patch_prober.go:29] interesting pod/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.142:15021/healthz/ready\": dial tcp 10.133.0.142:15021: connect: connection refused" start-of-body=
Oct 30 22:13:14.641681 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:14.641130    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w" podUID="73ac22d3-ce8c-4c0c-a742-11d19526d964" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.142:15021/healthz/ready\": dial tcp 10.133.0.142:15021: connect: connection refused"
Oct 30 22:13:15.641352 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:15.641305    4871 patch_prober.go:29] interesting pod/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.142:15021/healthz/ready\": dial tcp 10.133.0.142:15021: connect: connection refused" start-of-body=
Oct 30 22:13:15.641810 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:15.641387    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w" podUID="73ac22d3-ce8c-4c0c-a742-11d19526d964" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.142:15021/healthz/ready\": dial tcp 10.133.0.142:15021: connect: connection refused"
Oct 30 22:13:15.663882 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:15.663833    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"
Oct 30 22:13:16.117293 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:16.117195    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:16.633056 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:16.633008    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"]
Oct 30 22:13:16.636262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:16.634060    4871 cpu_manager.go:414] "containerMap: removing container" podUID="8937fa65-1fb5-4614-9be1-9075e2b3f46b" containerName="istio-proxy"
Oct 30 22:13:16.636262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:16.634088    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8937fa65-1fb5-4614-9be1-9075e2b3f46b" containerName="istio-proxy"
Oct 30 22:13:16.636262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:16.634285    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="8937fa65-1fb5-4614-9be1-9075e2b3f46b" containerName="istio-proxy"
Oct 30 22:13:16.652575 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:16.652532    4871 patch_prober.go:29] interesting pod/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.142:15021/healthz/ready\": dial tcp 10.133.0.142:15021: connect: connection refused" start-of-body=
Oct 30 22:13:16.653209 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:16.653172    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w" podUID="73ac22d3-ce8c-4c0c-a742-11d19526d964" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.142:15021/healthz/ready\": dial tcp 10.133.0.142:15021: connect: connection refused"
Oct 30 22:13:16.660300 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:16.653949    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"
Oct 30 22:13:16.710261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:16.705027    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"]
Oct 30 22:13:16.753259 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:16.737070    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6674b9d4-017b-4109-9262-bc8ad1244712-workload-certs\") pod \"e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856\" (UID: \"6674b9d4-017b-4109-9262-bc8ad1244712\") " pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"
Oct 30 22:13:16.764409 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:16.764372    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6674b9d4-017b-4109-9262-bc8ad1244712-istio-envoy\") pod \"e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856\" (UID: \"6674b9d4-017b-4109-9262-bc8ad1244712\") " pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"
Oct 30 22:13:16.774447 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:16.774404    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6674b9d4-017b-4109-9262-bc8ad1244712-istio-podinfo\") pod \"e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856\" (UID: \"6674b9d4-017b-4109-9262-bc8ad1244712\") " pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"
Oct 30 22:13:16.814266 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:16.796702    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6674b9d4-017b-4109-9262-bc8ad1244712-istio-data\") pod \"e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856\" (UID: \"6674b9d4-017b-4109-9262-bc8ad1244712\") " pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"
Oct 30 22:13:16.814665 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:16.814630    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6674b9d4-017b-4109-9262-bc8ad1244712-istiod-ca-cert\") pod \"e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856\" (UID: \"6674b9d4-017b-4109-9262-bc8ad1244712\") " pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"
Oct 30 22:13:16.814879 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:16.814863    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6674b9d4-017b-4109-9262-bc8ad1244712-workload-socket\") pod \"e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856\" (UID: \"6674b9d4-017b-4109-9262-bc8ad1244712\") " pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"
Oct 30 22:13:16.826266 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:16.815144    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6674b9d4-017b-4109-9262-bc8ad1244712-istio-token\") pod \"e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856\" (UID: \"6674b9d4-017b-4109-9262-bc8ad1244712\") " pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"
Oct 30 22:13:16.890487 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:16.850369    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-c4bfac-openshift-default-dockercfg-zmn9k\""
Oct 30 22:13:16.890487 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:16.868385    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:16.916837 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:16.916762    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66gcx\" (UniqueName: \"kubernetes.io/projected/6674b9d4-017b-4109-9262-bc8ad1244712-kube-api-access-66gcx\") pod \"e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856\" (UID: \"6674b9d4-017b-4109-9262-bc8ad1244712\") " pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"
Oct 30 22:13:16.942065 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:16.917312    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6674b9d4-017b-4109-9262-bc8ad1244712-credential-socket\") pod \"e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856\" (UID: \"6674b9d4-017b-4109-9262-bc8ad1244712\") " pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"
Oct 30 22:13:17.058491 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:17.048023    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6674b9d4-017b-4109-9262-bc8ad1244712-workload-socket\") pod \"e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856\" (UID: \"6674b9d4-017b-4109-9262-bc8ad1244712\") " pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"
Oct 30 22:13:17.058491 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:17.051987    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6674b9d4-017b-4109-9262-bc8ad1244712-workload-socket\") pod \"e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856\" (UID: \"6674b9d4-017b-4109-9262-bc8ad1244712\") " pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"
Oct 30 22:13:17.058491 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:17.056692    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6674b9d4-017b-4109-9262-bc8ad1244712-istio-token\") pod \"e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856\" (UID: \"6674b9d4-017b-4109-9262-bc8ad1244712\") " pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"
Oct 30 22:13:17.058491 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:17.056747    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66gcx\" (UniqueName: \"kubernetes.io/projected/6674b9d4-017b-4109-9262-bc8ad1244712-kube-api-access-66gcx\") pod \"e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856\" (UID: \"6674b9d4-017b-4109-9262-bc8ad1244712\") " pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"
Oct 30 22:13:17.058491 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:17.056786    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6674b9d4-017b-4109-9262-bc8ad1244712-credential-socket\") pod \"e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856\" (UID: \"6674b9d4-017b-4109-9262-bc8ad1244712\") " pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"
Oct 30 22:13:17.058491 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:17.057009    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6674b9d4-017b-4109-9262-bc8ad1244712-workload-certs\") pod \"e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856\" (UID: \"6674b9d4-017b-4109-9262-bc8ad1244712\") " pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"
Oct 30 22:13:17.058491 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:17.057065    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6674b9d4-017b-4109-9262-bc8ad1244712-istio-envoy\") pod \"e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856\" (UID: \"6674b9d4-017b-4109-9262-bc8ad1244712\") " pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"
Oct 30 22:13:17.058491 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:17.057093    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6674b9d4-017b-4109-9262-bc8ad1244712-istio-podinfo\") pod \"e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856\" (UID: \"6674b9d4-017b-4109-9262-bc8ad1244712\") " pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"
Oct 30 22:13:17.058491 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:17.057132    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6674b9d4-017b-4109-9262-bc8ad1244712-istio-data\") pod \"e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856\" (UID: \"6674b9d4-017b-4109-9262-bc8ad1244712\") " pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"
Oct 30 22:13:17.058491 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:17.057172    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6674b9d4-017b-4109-9262-bc8ad1244712-istiod-ca-cert\") pod \"e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856\" (UID: \"6674b9d4-017b-4109-9262-bc8ad1244712\") " pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"
Oct 30 22:13:17.058491 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:17.057740    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6674b9d4-017b-4109-9262-bc8ad1244712-workload-certs\") pod \"e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856\" (UID: \"6674b9d4-017b-4109-9262-bc8ad1244712\") " pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"
Oct 30 22:13:17.058491 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:17.058030    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6674b9d4-017b-4109-9262-bc8ad1244712-credential-socket\") pod \"e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856\" (UID: \"6674b9d4-017b-4109-9262-bc8ad1244712\") " pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"
Oct 30 22:13:17.058491 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:17.058373    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6674b9d4-017b-4109-9262-bc8ad1244712-istiod-ca-cert\") pod \"e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856\" (UID: \"6674b9d4-017b-4109-9262-bc8ad1244712\") " pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"
Oct 30 22:13:17.058491 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:17.058385    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6674b9d4-017b-4109-9262-bc8ad1244712-istio-data\") pod \"e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856\" (UID: \"6674b9d4-017b-4109-9262-bc8ad1244712\") " pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"
Oct 30 22:13:17.060702 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:17.060666    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:17.061531 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:17.061508    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6674b9d4-017b-4109-9262-bc8ad1244712-istio-envoy\") pod \"e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856\" (UID: \"6674b9d4-017b-4109-9262-bc8ad1244712\") " pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"
Oct 30 22:13:17.062158 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:17.062137    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6674b9d4-017b-4109-9262-bc8ad1244712-istio-podinfo\") pod \"e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856\" (UID: \"6674b9d4-017b-4109-9262-bc8ad1244712\") " pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"
Oct 30 22:13:17.135992 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:17.135948    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6674b9d4-017b-4109-9262-bc8ad1244712-istio-token\") pod \"e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856\" (UID: \"6674b9d4-017b-4109-9262-bc8ad1244712\") " pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"
Oct 30 22:13:17.138293 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:17.138222    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-66gcx\" (UniqueName: \"kubernetes.io/projected/6674b9d4-017b-4109-9262-bc8ad1244712-kube-api-access-66gcx\") pod \"e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856\" (UID: \"6674b9d4-017b-4109-9262-bc8ad1244712\") " pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"
Oct 30 22:13:17.316447 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:17.316325    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"
Oct 30 22:13:17.647979 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:17.647934    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"
Oct 30 22:13:17.649206 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:17.649179    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"
Oct 30 22:13:17.770553 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:17.764489    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"]
Oct 30 22:13:17.834936 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:17.832494    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:13:17.834936 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:17.832601    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:13:17.834936 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:17.832643    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:13:18.043128 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:18.043070    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856" event={"ID":"6674b9d4-017b-4109-9262-bc8ad1244712","Type":"ContainerStarted","Data":"7b93e238cb14ccef31ed03ca1a25e2ede53c102435d6fb5fe8b45a2c87af535b"}
Oct 30 22:13:18.043381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:18.043161    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856" event={"ID":"6674b9d4-017b-4109-9262-bc8ad1244712","Type":"ContainerStarted","Data":"ba9c1c10322b8f7bc663355853ecf0824d275a080ef2d0700a10e67859bb47f5"}
Oct 30 22:13:18.160429 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:18.160320    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856" podStartSLOduration=2.160304604 podStartE2EDuration="2.160304604s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:13:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:13:18.157936191 +0000 UTC m=+2140.335424808" watchObservedRunningTime="2026-10-30 22:13:18.160304604 +0000 UTC m=+2140.337793222"
Oct 30 22:13:18.321224 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:18.321185    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"
Oct 30 22:13:18.321657 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:18.321628    4871 patch_prober.go:29] interesting pod/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused" start-of-body=
Oct 30 22:13:18.321857 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:18.321833    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856" podUID="6674b9d4-017b-4109-9262-bc8ad1244712" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused"
Oct 30 22:13:18.626072 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:18.626030    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:19.322162 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:19.322122    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"
Oct 30 22:13:20.059151 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:20.059118    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"
Oct 30 22:13:20.060176 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:20.060152    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"
Oct 30 22:13:20.993863 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:20.993818    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:21.194736 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:21.194700    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:21.382361 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:21.382323    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:23.159090 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:23.159046    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:23.755692 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:23.755654    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:24.074745 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:24.074662    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:24.939623 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:24.938143    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:25.094192 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:25.094152    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:25.458259 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:25.454781    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:25.621044 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:25.620911    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:29.142985 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:29.142950    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:29.324761 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:29.324709    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:29.499080 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:29.498906    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:30.986011 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:30.985977    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:31.185039 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:31.185002    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:31.372228 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:31.372193    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:34.352613 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:34.352572    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:35.569621 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:13:35.569583    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:13:35.710841 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:35.710804    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:36.563269 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:36.563201    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:36.769502 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:36.768411    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:37.002315 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:37.002260    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:37.550945 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:37.550906    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"]
Oct 30 22:13:37.554640 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:37.554615    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"
Oct 30 22:13:37.604420 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:37.604382    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-138566-openshift-default-dockercfg-mcjlm\""
Oct 30 22:13:37.640701 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:37.640651    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a9121ce-2d96-4f01-8463-ef1768b5f588-istio-token\") pod \"e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4\" (UID: \"4a9121ce-2d96-4f01-8463-ef1768b5f588\") " pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"
Oct 30 22:13:37.640937 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:37.640911    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a9121ce-2d96-4f01-8463-ef1768b5f588-istio-podinfo\") pod \"e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4\" (UID: \"4a9121ce-2d96-4f01-8463-ef1768b5f588\") " pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"
Oct 30 22:13:37.641064 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:37.641040    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a9121ce-2d96-4f01-8463-ef1768b5f588-istiod-ca-cert\") pod \"e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4\" (UID: \"4a9121ce-2d96-4f01-8463-ef1768b5f588\") " pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"
Oct 30 22:13:37.641132 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:37.641113    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a9121ce-2d96-4f01-8463-ef1768b5f588-istio-envoy\") pod \"e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4\" (UID: \"4a9121ce-2d96-4f01-8463-ef1768b5f588\") " pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"
Oct 30 22:13:37.641192 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:37.641154    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdlb5\" (UniqueName: \"kubernetes.io/projected/4a9121ce-2d96-4f01-8463-ef1768b5f588-kube-api-access-vdlb5\") pod \"e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4\" (UID: \"4a9121ce-2d96-4f01-8463-ef1768b5f588\") " pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"
Oct 30 22:13:37.641270 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:37.641188    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a9121ce-2d96-4f01-8463-ef1768b5f588-istio-data\") pod \"e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4\" (UID: \"4a9121ce-2d96-4f01-8463-ef1768b5f588\") " pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"
Oct 30 22:13:37.641339 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:37.641268    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a9121ce-2d96-4f01-8463-ef1768b5f588-credential-socket\") pod \"e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4\" (UID: \"4a9121ce-2d96-4f01-8463-ef1768b5f588\") " pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"
Oct 30 22:13:37.641339 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:37.641294    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a9121ce-2d96-4f01-8463-ef1768b5f588-workload-certs\") pod \"e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4\" (UID: \"4a9121ce-2d96-4f01-8463-ef1768b5f588\") " pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"
Oct 30 22:13:37.641447 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:37.641381    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a9121ce-2d96-4f01-8463-ef1768b5f588-workload-socket\") pod \"e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4\" (UID: \"4a9121ce-2d96-4f01-8463-ef1768b5f588\") " pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"
Oct 30 22:13:37.658504 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:37.658464    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"]
Oct 30 22:13:37.662709 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:37.662679    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:37.752264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:37.745569    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a9121ce-2d96-4f01-8463-ef1768b5f588-istio-data\") pod \"e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4\" (UID: \"4a9121ce-2d96-4f01-8463-ef1768b5f588\") " pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"
Oct 30 22:13:37.752264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:37.745646    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a9121ce-2d96-4f01-8463-ef1768b5f588-credential-socket\") pod \"e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4\" (UID: \"4a9121ce-2d96-4f01-8463-ef1768b5f588\") " pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"
Oct 30 22:13:37.752264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:37.745675    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a9121ce-2d96-4f01-8463-ef1768b5f588-workload-certs\") pod \"e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4\" (UID: \"4a9121ce-2d96-4f01-8463-ef1768b5f588\") " pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"
Oct 30 22:13:37.752264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:37.745765    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a9121ce-2d96-4f01-8463-ef1768b5f588-workload-socket\") pod \"e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4\" (UID: \"4a9121ce-2d96-4f01-8463-ef1768b5f588\") " pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"
Oct 30 22:13:37.752264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:37.745819    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a9121ce-2d96-4f01-8463-ef1768b5f588-istio-token\") pod \"e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4\" (UID: \"4a9121ce-2d96-4f01-8463-ef1768b5f588\") " pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"
Oct 30 22:13:37.752264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:37.745975    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a9121ce-2d96-4f01-8463-ef1768b5f588-istio-podinfo\") pod \"e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4\" (UID: \"4a9121ce-2d96-4f01-8463-ef1768b5f588\") " pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"
Oct 30 22:13:37.752264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:37.746028    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a9121ce-2d96-4f01-8463-ef1768b5f588-istiod-ca-cert\") pod \"e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4\" (UID: \"4a9121ce-2d96-4f01-8463-ef1768b5f588\") " pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"
Oct 30 22:13:37.752264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:37.746087    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a9121ce-2d96-4f01-8463-ef1768b5f588-istio-envoy\") pod \"e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4\" (UID: \"4a9121ce-2d96-4f01-8463-ef1768b5f588\") " pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"
Oct 30 22:13:37.752264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:37.746116    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdlb5\" (UniqueName: \"kubernetes.io/projected/4a9121ce-2d96-4f01-8463-ef1768b5f588-kube-api-access-vdlb5\") pod \"e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4\" (UID: \"4a9121ce-2d96-4f01-8463-ef1768b5f588\") " pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"
Oct 30 22:13:37.752264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:37.746969    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a9121ce-2d96-4f01-8463-ef1768b5f588-istio-data\") pod \"e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4\" (UID: \"4a9121ce-2d96-4f01-8463-ef1768b5f588\") " pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"
Oct 30 22:13:37.752264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:37.747216    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a9121ce-2d96-4f01-8463-ef1768b5f588-credential-socket\") pod \"e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4\" (UID: \"4a9121ce-2d96-4f01-8463-ef1768b5f588\") " pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"
Oct 30 22:13:37.752264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:37.747490    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a9121ce-2d96-4f01-8463-ef1768b5f588-workload-certs\") pod \"e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4\" (UID: \"4a9121ce-2d96-4f01-8463-ef1768b5f588\") " pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"
Oct 30 22:13:37.752264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:37.747770    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a9121ce-2d96-4f01-8463-ef1768b5f588-workload-socket\") pod \"e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4\" (UID: \"4a9121ce-2d96-4f01-8463-ef1768b5f588\") " pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"
Oct 30 22:13:37.759313 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:37.757120    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a9121ce-2d96-4f01-8463-ef1768b5f588-istiod-ca-cert\") pod \"e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4\" (UID: \"4a9121ce-2d96-4f01-8463-ef1768b5f588\") " pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"
Oct 30 22:13:37.779291 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:37.779255    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a9121ce-2d96-4f01-8463-ef1768b5f588-istio-podinfo\") pod \"e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4\" (UID: \"4a9121ce-2d96-4f01-8463-ef1768b5f588\") " pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"
Oct 30 22:13:37.782092 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:37.782060    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a9121ce-2d96-4f01-8463-ef1768b5f588-istio-envoy\") pod \"e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4\" (UID: \"4a9121ce-2d96-4f01-8463-ef1768b5f588\") " pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"
Oct 30 22:13:37.808147 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:37.808070    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdlb5\" (UniqueName: \"kubernetes.io/projected/4a9121ce-2d96-4f01-8463-ef1768b5f588-kube-api-access-vdlb5\") pod \"e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4\" (UID: \"4a9121ce-2d96-4f01-8463-ef1768b5f588\") " pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"
Oct 30 22:13:37.808452 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:37.808427    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a9121ce-2d96-4f01-8463-ef1768b5f588-istio-token\") pod \"e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4\" (UID: \"4a9121ce-2d96-4f01-8463-ef1768b5f588\") " pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"
Oct 30 22:13:37.868309 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:37.868272    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"
Oct 30 22:13:37.905560 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:37.905497    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:38.225265 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:38.222883    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:38.309436 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:38.309403    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"]
Oct 30 22:13:38.310273 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:13:38.310207    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a9121ce_2d96_4f01_8463_ef1768b5f588.slice/crio-245f858eb2b097ceef474e1ef699e2f44278dd169f3fda48be7288ce66536ce5 WatchSource:0}: Error finding container 245f858eb2b097ceef474e1ef699e2f44278dd169f3fda48be7288ce66536ce5: Status 404 returned error can't find the container with id 245f858eb2b097ceef474e1ef699e2f44278dd169f3fda48be7288ce66536ce5
Oct 30 22:13:38.313131 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:38.313096    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:13:38.313265 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:38.313175    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:13:38.313265 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:38.313221    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:13:38.393084 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:38.390704    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:38.452327 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:38.452277    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4" event={"ID":"4a9121ce-2d96-4f01-8463-ef1768b5f588","Type":"ContainerStarted","Data":"25b482f2f890c986648d162aa82289d0f3e199d77e465e88f36e4bc5830d74fd"}
Oct 30 22:13:38.452522 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:38.452392    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4" event={"ID":"4a9121ce-2d96-4f01-8463-ef1768b5f588","Type":"ContainerStarted","Data":"245f858eb2b097ceef474e1ef699e2f44278dd169f3fda48be7288ce66536ce5"}
Oct 30 22:13:38.570396 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:38.570284    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4" podStartSLOduration=1.570269026 podStartE2EDuration="1.570269026s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:13:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:13:38.567187164 +0000 UTC m=+2160.744675780" watchObservedRunningTime="2026-10-30 22:13:38.570269026 +0000 UTC m=+2160.747757643"
Oct 30 22:13:38.868824 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:38.868795    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"
Oct 30 22:13:39.356312 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:39.354091    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:39.513258 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:39.510995    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:39.874785 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:39.874755    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"
Oct 30 22:13:40.496845 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:40.496816    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"
Oct 30 22:13:40.497820 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:40.497799    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"
Oct 30 22:13:40.892518 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:40.892483    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:41.970595 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:41.970562    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:42.144097 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:42.144040    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:42.301256 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:42.301157    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:45.194554 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:45.194508    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:45.402165 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:45.402130    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:46.292317 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:46.292278    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:51.210837 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:51.210801    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:56.250210 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:56.250167    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:56.845919 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:56.845882    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:57.650772 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:57.641587    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:57.826896 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:13:57.826859    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:01.862444 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:01.862403    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz"]
Oct 30 22:14:01.863030 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:01.862689    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz" podUID="eb723e66-473c-407f-ad2f-bbd77f86a6bf" containerName="maas-api" containerID="cri-o://8ceee39c5a91ebbb6902a0f59aca868021c4df57a05eeb6f1cb2f342a684103a" gracePeriod=30
Oct 30 22:14:02.003968 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:02.003896    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:02.216497 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:02.216460    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:02.224587 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:02.224559    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz"
Oct 30 22:14:02.340169 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:02.340132    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb723e66-473c-407f-ad2f-bbd77f86a6bf-kube-api-access-n98sn\" (UniqueName: \"kubernetes.io/projected/eb723e66-473c-407f-ad2f-bbd77f86a6bf-kube-api-access-n98sn\") pod \"eb723e66-473c-407f-ad2f-bbd77f86a6bf\" (UID: \"eb723e66-473c-407f-ad2f-bbd77f86a6bf\") "
Oct 30 22:14:02.340450 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:02.340191    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/eb723e66-473c-407f-ad2f-bbd77f86a6bf-metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb723e66-473c-407f-ad2f-bbd77f86a6bf-metrics-certs\") pod \"eb723e66-473c-407f-ad2f-bbd77f86a6bf\" (UID: \"eb723e66-473c-407f-ad2f-bbd77f86a6bf\") "
Oct 30 22:14:02.340522 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:02.340490    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/eb723e66-473c-407f-ad2f-bbd77f86a6bf-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb723e66-473c-407f-ad2f-bbd77f86a6bf-maas-api-tls\") pod \"eb723e66-473c-407f-ad2f-bbd77f86a6bf\" (UID: \"eb723e66-473c-407f-ad2f-bbd77f86a6bf\") "
Oct 30 22:14:02.340585 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:02.340521    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eb723e66-473c-407f-ad2f-bbd77f86a6bf-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb723e66-473c-407f-ad2f-bbd77f86a6bf-openshift-service-ca\") pod \"eb723e66-473c-407f-ad2f-bbd77f86a6bf\" (UID: \"eb723e66-473c-407f-ad2f-bbd77f86a6bf\") "
Oct 30 22:14:02.341540 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:02.341049    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb723e66-473c-407f-ad2f-bbd77f86a6bf-openshift-service-ca" pod "eb723e66-473c-407f-ad2f-bbd77f86a6bf" (UID: "eb723e66-473c-407f-ad2f-bbd77f86a6bf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:14:02.343875 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:02.343833    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb723e66-473c-407f-ad2f-bbd77f86a6bf-metrics-certs" pod "eb723e66-473c-407f-ad2f-bbd77f86a6bf" (UID: "eb723e66-473c-407f-ad2f-bbd77f86a6bf"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:14:02.344186 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:02.344161    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb723e66-473c-407f-ad2f-bbd77f86a6bf-maas-api-tls" pod "eb723e66-473c-407f-ad2f-bbd77f86a6bf" (UID: "eb723e66-473c-407f-ad2f-bbd77f86a6bf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:14:02.344772 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:02.344744    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb723e66-473c-407f-ad2f-bbd77f86a6bf-kube-api-access-n98sn" pod "eb723e66-473c-407f-ad2f-bbd77f86a6bf" (UID: "eb723e66-473c-407f-ad2f-bbd77f86a6bf"). InnerVolumeSpecName "kube-api-access-n98sn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:14:02.442515 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:02.442387    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb723e66-473c-407f-ad2f-bbd77f86a6bf-maas-api-tls\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:02.442515 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:02.442451    4871 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb723e66-473c-407f-ad2f-bbd77f86a6bf-openshift-service-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:02.442515 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:02.442466    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n98sn\" (UniqueName: \"kubernetes.io/projected/eb723e66-473c-407f-ad2f-bbd77f86a6bf-kube-api-access-n98sn\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:02.442515 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:02.442481    4871 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb723e66-473c-407f-ad2f-bbd77f86a6bf-metrics-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:02.757456 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:02.757415    4871 generic.go:356] "Generic (PLEG): container finished" podID="eb723e66-473c-407f-ad2f-bbd77f86a6bf" containerID="8ceee39c5a91ebbb6902a0f59aca868021c4df57a05eeb6f1cb2f342a684103a" exitCode=0
Oct 30 22:14:02.757456 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:02.757461    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz"
Oct 30 22:14:02.757727 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:02.757497    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz" event={"ID":"eb723e66-473c-407f-ad2f-bbd77f86a6bf","Type":"ContainerDied","Data":"8ceee39c5a91ebbb6902a0f59aca868021c4df57a05eeb6f1cb2f342a684103a"}
Oct 30 22:14:02.757727 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:02.757564    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz" event={"ID":"eb723e66-473c-407f-ad2f-bbd77f86a6bf","Type":"ContainerDied","Data":"a667d62e0d323e14e2d3e4049262be51d4d3a46418505cd2d0d39ab9d0e37bdc"}
Oct 30 22:14:02.757727 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:02.757590    4871 scope.go:122] "RemoveContainer" containerID="8ceee39c5a91ebbb6902a0f59aca868021c4df57a05eeb6f1cb2f342a684103a"
Oct 30 22:14:02.769337 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:02.769310    4871 scope.go:122] "RemoveContainer" containerID="8ceee39c5a91ebbb6902a0f59aca868021c4df57a05eeb6f1cb2f342a684103a"
Oct 30 22:14:02.769707 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:14:02.769686    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ceee39c5a91ebbb6902a0f59aca868021c4df57a05eeb6f1cb2f342a684103a\": container with ID starting with 8ceee39c5a91ebbb6902a0f59aca868021c4df57a05eeb6f1cb2f342a684103a not found: ID does not exist" containerID="8ceee39c5a91ebbb6902a0f59aca868021c4df57a05eeb6f1cb2f342a684103a"
Oct 30 22:14:02.769768 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:02.769717    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ceee39c5a91ebbb6902a0f59aca868021c4df57a05eeb6f1cb2f342a684103a"} err="failed to get container status \"8ceee39c5a91ebbb6902a0f59aca868021c4df57a05eeb6f1cb2f342a684103a\": rpc error: code = NotFound desc = could not find container \"8ceee39c5a91ebbb6902a0f59aca868021c4df57a05eeb6f1cb2f342a684103a\": container with ID starting with 8ceee39c5a91ebbb6902a0f59aca868021c4df57a05eeb6f1cb2f342a684103a not found: ID does not exist"
Oct 30 22:14:02.952339 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:02.952291    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz"]
Oct 30 22:14:02.963260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:02.957163    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-52f191-847d9dcd65-k7nlz"]
Oct 30 22:14:03.677930 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:03.677895    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:03.888337 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:03.888298    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:03.965214 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:03.965110    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eb723e66-473c-407f-ad2f-bbd77f86a6bf" path="/var/lib/kubelet/pods/eb723e66-473c-407f-ad2f-bbd77f86a6bf/volumes"
Oct 30 22:14:04.045913 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:04.045876    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:04.597583 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:04.597534    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:05.570445 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:14:05.570403    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:14:06.635864 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:06.635821    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:06.881310 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:06.870381    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:08.885282 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:08.884050    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:10.911110 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:10.911074    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:11.112503 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:11.112463    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:11.253266 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:11.253168    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"]
Oct 30 22:14:11.253530 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:11.253492    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh" podUID="088f4439-453d-406a-b54a-837e10b36efd" containerName="istio-proxy" containerID="cri-o://6d11c99737badb09049032029ea5a0df9f82d701d2535c86fccbcd4089bb88ed" gracePeriod=30
Oct 30 22:14:11.263916 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:11.263887    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:14.141053 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:14.141002    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:16.536458 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:16.536375    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"
Oct 30 22:14:16.597104 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:16.597066    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/088f4439-453d-406a-b54a-837e10b36efd-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/088f4439-453d-406a-b54a-837e10b36efd-istio-data\") pod \"088f4439-453d-406a-b54a-837e10b36efd\" (UID: \"088f4439-453d-406a-b54a-837e10b36efd\") "
Oct 30 22:14:16.597334 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:16.597180    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/088f4439-453d-406a-b54a-837e10b36efd-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/088f4439-453d-406a-b54a-837e10b36efd-istiod-ca-cert\") pod \"088f4439-453d-406a-b54a-837e10b36efd\" (UID: \"088f4439-453d-406a-b54a-837e10b36efd\") "
Oct 30 22:14:16.597334 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:16.597312    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/088f4439-453d-406a-b54a-837e10b36efd-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/088f4439-453d-406a-b54a-837e10b36efd-workload-certs\") pod \"088f4439-453d-406a-b54a-837e10b36efd\" (UID: \"088f4439-453d-406a-b54a-837e10b36efd\") "
Oct 30 22:14:16.597468 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:16.597369    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/088f4439-453d-406a-b54a-837e10b36efd-istio-token\" (UniqueName: \"kubernetes.io/projected/088f4439-453d-406a-b54a-837e10b36efd-istio-token\") pod \"088f4439-453d-406a-b54a-837e10b36efd\" (UID: \"088f4439-453d-406a-b54a-837e10b36efd\") "
Oct 30 22:14:16.597533 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:16.597462    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/088f4439-453d-406a-b54a-837e10b36efd-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/088f4439-453d-406a-b54a-837e10b36efd-workload-socket\") pod \"088f4439-453d-406a-b54a-837e10b36efd\" (UID: \"088f4439-453d-406a-b54a-837e10b36efd\") "
Oct 30 22:14:16.597588 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:16.597545    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/088f4439-453d-406a-b54a-837e10b36efd-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/088f4439-453d-406a-b54a-837e10b36efd-istio-envoy\") pod \"088f4439-453d-406a-b54a-837e10b36efd\" (UID: \"088f4439-453d-406a-b54a-837e10b36efd\") "
Oct 30 22:14:16.597643 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:16.597456    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/088f4439-453d-406a-b54a-837e10b36efd-istio-data" pod "088f4439-453d-406a-b54a-837e10b36efd" (UID: "088f4439-453d-406a-b54a-837e10b36efd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:16.597643 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:16.597576    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/088f4439-453d-406a-b54a-837e10b36efd-istiod-ca-cert" pod "088f4439-453d-406a-b54a-837e10b36efd" (UID: "088f4439-453d-406a-b54a-837e10b36efd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:14:16.597643 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:16.597592    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/088f4439-453d-406a-b54a-837e10b36efd-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/088f4439-453d-406a-b54a-837e10b36efd-credential-socket\") pod \"088f4439-453d-406a-b54a-837e10b36efd\" (UID: \"088f4439-453d-406a-b54a-837e10b36efd\") "
Oct 30 22:14:16.597643 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:16.597627    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/088f4439-453d-406a-b54a-837e10b36efd-workload-certs" pod "088f4439-453d-406a-b54a-837e10b36efd" (UID: "088f4439-453d-406a-b54a-837e10b36efd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:16.597800 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:16.597665    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/088f4439-453d-406a-b54a-837e10b36efd-workload-socket" pod "088f4439-453d-406a-b54a-837e10b36efd" (UID: "088f4439-453d-406a-b54a-837e10b36efd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:16.597800 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:16.597675    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/088f4439-453d-406a-b54a-837e10b36efd-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/088f4439-453d-406a-b54a-837e10b36efd-istio-podinfo\") pod \"088f4439-453d-406a-b54a-837e10b36efd\" (UID: \"088f4439-453d-406a-b54a-837e10b36efd\") "
Oct 30 22:14:16.597800 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:16.597716    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/088f4439-453d-406a-b54a-837e10b36efd-kube-api-access-sx27c\" (UniqueName: \"kubernetes.io/projected/088f4439-453d-406a-b54a-837e10b36efd-kube-api-access-sx27c\") pod \"088f4439-453d-406a-b54a-837e10b36efd\" (UID: \"088f4439-453d-406a-b54a-837e10b36efd\") "
Oct 30 22:14:16.597957 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:16.597799    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/088f4439-453d-406a-b54a-837e10b36efd-credential-socket" pod "088f4439-453d-406a-b54a-837e10b36efd" (UID: "088f4439-453d-406a-b54a-837e10b36efd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:16.598226 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:16.598198    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/088f4439-453d-406a-b54a-837e10b36efd-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:16.598349 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:16.598228    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/088f4439-453d-406a-b54a-837e10b36efd-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:16.598349 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:16.598271    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/088f4439-453d-406a-b54a-837e10b36efd-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:16.598349 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:16.598283    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/088f4439-453d-406a-b54a-837e10b36efd-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:16.598349 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:16.598292    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/088f4439-453d-406a-b54a-837e10b36efd-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:16.599719 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:16.599694    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/088f4439-453d-406a-b54a-837e10b36efd-istio-token" pod "088f4439-453d-406a-b54a-837e10b36efd" (UID: "088f4439-453d-406a-b54a-837e10b36efd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:14:16.600101 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:16.600068    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/088f4439-453d-406a-b54a-837e10b36efd-istio-envoy" pod "088f4439-453d-406a-b54a-837e10b36efd" (UID: "088f4439-453d-406a-b54a-837e10b36efd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:16.600174 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:16.600102    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/088f4439-453d-406a-b54a-837e10b36efd-istio-podinfo" pod "088f4439-453d-406a-b54a-837e10b36efd" (UID: "088f4439-453d-406a-b54a-837e10b36efd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:14:16.600174 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:16.600142    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/088f4439-453d-406a-b54a-837e10b36efd-kube-api-access-sx27c" pod "088f4439-453d-406a-b54a-837e10b36efd" (UID: "088f4439-453d-406a-b54a-837e10b36efd"). InnerVolumeSpecName "kube-api-access-sx27c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:14:16.699116 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:16.699082    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/088f4439-453d-406a-b54a-837e10b36efd-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:16.699116 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:16.699116    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/088f4439-453d-406a-b54a-837e10b36efd-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:16.699116 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:16.699127    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sx27c\" (UniqueName: \"kubernetes.io/projected/088f4439-453d-406a-b54a-837e10b36efd-kube-api-access-sx27c\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:16.699453 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:16.699137    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/088f4439-453d-406a-b54a-837e10b36efd-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:14:17.065311 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:17.065271    4871 generic.go:356] "Generic (PLEG): container finished" podID="088f4439-453d-406a-b54a-837e10b36efd" containerID="6d11c99737badb09049032029ea5a0df9f82d701d2535c86fccbcd4089bb88ed" exitCode=0
Oct 30 22:14:17.065311 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:17.065314    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"
Oct 30 22:14:17.065555 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:17.065333    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh" event={"ID":"088f4439-453d-406a-b54a-837e10b36efd","Type":"ContainerDied","Data":"6d11c99737badb09049032029ea5a0df9f82d701d2535c86fccbcd4089bb88ed"}
Oct 30 22:14:17.065555 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:17.065406    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh" event={"ID":"088f4439-453d-406a-b54a-837e10b36efd","Type":"ContainerDied","Data":"b59d088c89d5fe943b57089bbaa55fad9b2ac2cda86b56f91670f5ca673b74fe"}
Oct 30 22:14:17.065555 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:17.065439    4871 scope.go:122] "RemoveContainer" containerID="6d11c99737badb09049032029ea5a0df9f82d701d2535c86fccbcd4089bb88ed"
Oct 30 22:14:17.079719 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:17.079699    4871 scope.go:122] "RemoveContainer" containerID="df09c5f2757e1fed96666c817850397aac172e4d3efb7e932f8e6439ae47cc98"
Oct 30 22:14:17.089775 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:17.089753    4871 scope.go:122] "RemoveContainer" containerID="6d11c99737badb09049032029ea5a0df9f82d701d2535c86fccbcd4089bb88ed"
Oct 30 22:14:17.090049 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:14:17.090031    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d11c99737badb09049032029ea5a0df9f82d701d2535c86fccbcd4089bb88ed\": container with ID starting with 6d11c99737badb09049032029ea5a0df9f82d701d2535c86fccbcd4089bb88ed not found: ID does not exist" containerID="6d11c99737badb09049032029ea5a0df9f82d701d2535c86fccbcd4089bb88ed"
Oct 30 22:14:17.090091 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:17.090059    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d11c99737badb09049032029ea5a0df9f82d701d2535c86fccbcd4089bb88ed"} err="failed to get container status \"6d11c99737badb09049032029ea5a0df9f82d701d2535c86fccbcd4089bb88ed\": rpc error: code = NotFound desc = could not find container \"6d11c99737badb09049032029ea5a0df9f82d701d2535c86fccbcd4089bb88ed\": container with ID starting with 6d11c99737badb09049032029ea5a0df9f82d701d2535c86fccbcd4089bb88ed not found: ID does not exist"
Oct 30 22:14:17.090091 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:17.090077    4871 scope.go:122] "RemoveContainer" containerID="df09c5f2757e1fed96666c817850397aac172e4d3efb7e932f8e6439ae47cc98"
Oct 30 22:14:17.090347 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:14:17.090328    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df09c5f2757e1fed96666c817850397aac172e4d3efb7e932f8e6439ae47cc98\": container with ID starting with df09c5f2757e1fed96666c817850397aac172e4d3efb7e932f8e6439ae47cc98 not found: ID does not exist" containerID="df09c5f2757e1fed96666c817850397aac172e4d3efb7e932f8e6439ae47cc98"
Oct 30 22:14:17.090391 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:17.090351    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df09c5f2757e1fed96666c817850397aac172e4d3efb7e932f8e6439ae47cc98"} err="failed to get container status \"df09c5f2757e1fed96666c817850397aac172e4d3efb7e932f8e6439ae47cc98\": rpc error: code = NotFound desc = could not find container \"df09c5f2757e1fed96666c817850397aac172e4d3efb7e932f8e6439ae47cc98\": container with ID starting with df09c5f2757e1fed96666c817850397aac172e4d3efb7e932f8e6439ae47cc98 not found: ID does not exist"
Oct 30 22:14:17.235027 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:17.234989    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"]
Oct 30 22:14:17.282021 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:17.281989    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-52f191-openshift-default-6d97c4967c-gd5qh"]
Oct 30 22:14:17.807968 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:17.807929    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:17.962554 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:17.962479    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="088f4439-453d-406a-b54a-837e10b36efd" path="/var/lib/kubelet/pods/088f4439-453d-406a-b54a-837e10b36efd/volumes"
Oct 30 22:14:18.344685 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:18.344643    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:20.192334 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:20.192291    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a4257d-c6b8d878d-kr4t7"]
Oct 30 22:14:20.192940 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:20.192925    4871 cpu_manager.go:414] "containerMap: removing container" podUID="088f4439-453d-406a-b54a-837e10b36efd" containerName="istio-proxy"
Oct 30 22:14:20.192998 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:20.192941    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="088f4439-453d-406a-b54a-837e10b36efd" containerName="istio-proxy"
Oct 30 22:14:20.192998 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:20.192978    4871 cpu_manager.go:414] "containerMap: removing container" podUID="088f4439-453d-406a-b54a-837e10b36efd" containerName="istio-proxy"
Oct 30 22:14:20.192998 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:20.192984    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="088f4439-453d-406a-b54a-837e10b36efd" containerName="istio-proxy"
Oct 30 22:14:20.192998 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:20.192996    4871 cpu_manager.go:414] "containerMap: removing container" podUID="eb723e66-473c-407f-ad2f-bbd77f86a6bf" containerName="maas-api"
Oct 30 22:14:20.193149 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:20.193001    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eb723e66-473c-407f-ad2f-bbd77f86a6bf" containerName="maas-api"
Oct 30 22:14:20.193149 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:20.193078    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="088f4439-453d-406a-b54a-837e10b36efd" containerName="istio-proxy"
Oct 30 22:14:20.193149 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:20.193093    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="088f4439-453d-406a-b54a-837e10b36efd" containerName="istio-proxy"
Oct 30 22:14:20.193149 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:20.193100    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="eb723e66-473c-407f-ad2f-bbd77f86a6bf" containerName="maas-api"
Oct 30 22:14:20.198514 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:20.198491    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-a4257d-c6b8d878d-kr4t7"
Oct 30 22:14:20.252621 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:20.252583    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a4257d-c6b8d878d-kr4t7"]
Oct 30 22:14:20.260578 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:20.260539    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hp9t\" (UniqueName: \"kubernetes.io/projected/7d8bd465-4a7c-4480-9079-eac8aed7c7a2-kube-api-access-5hp9t\") pod \"payload-processing-e2e-ipp-a-a4257d-c6b8d878d-kr4t7\" (UID: \"7d8bd465-4a7c-4480-9079-eac8aed7c7a2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a4257d-c6b8d878d-kr4t7"
Oct 30 22:14:20.260761 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:20.260704    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d8bd465-4a7c-4480-9079-eac8aed7c7a2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-a4257d-c6b8d878d-kr4t7\" (UID: \"7d8bd465-4a7c-4480-9079-eac8aed7c7a2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a4257d-c6b8d878d-kr4t7"
Oct 30 22:14:20.294617 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:20.294569    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-a4257d\""
Oct 30 22:14:20.294800 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:20.294639    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-a4257d-dockercfg-ql9l2\""
Oct 30 22:14:20.362283 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:20.362216    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hp9t\" (UniqueName: \"kubernetes.io/projected/7d8bd465-4a7c-4480-9079-eac8aed7c7a2-kube-api-access-5hp9t\") pod \"payload-processing-e2e-ipp-a-a4257d-c6b8d878d-kr4t7\" (UID: \"7d8bd465-4a7c-4480-9079-eac8aed7c7a2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a4257d-c6b8d878d-kr4t7"
Oct 30 22:14:20.362480 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:20.362345    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d8bd465-4a7c-4480-9079-eac8aed7c7a2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-a4257d-c6b8d878d-kr4t7\" (UID: \"7d8bd465-4a7c-4480-9079-eac8aed7c7a2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a4257d-c6b8d878d-kr4t7"
Oct 30 22:14:20.363052 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:20.363031    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d8bd465-4a7c-4480-9079-eac8aed7c7a2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-a4257d-c6b8d878d-kr4t7\" (UID: \"7d8bd465-4a7c-4480-9079-eac8aed7c7a2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a4257d-c6b8d878d-kr4t7"
Oct 30 22:14:20.420783 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:20.420747    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hp9t\" (UniqueName: \"kubernetes.io/projected/7d8bd465-4a7c-4480-9079-eac8aed7c7a2-kube-api-access-5hp9t\") pod \"payload-processing-e2e-ipp-a-a4257d-c6b8d878d-kr4t7\" (UID: \"7d8bd465-4a7c-4480-9079-eac8aed7c7a2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a4257d-c6b8d878d-kr4t7"
Oct 30 22:14:20.513547 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:20.513443    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-a4257d-c6b8d878d-kr4t7"
Oct 30 22:14:20.888086 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:20.888052    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a4257d-c6b8d878d-kr4t7"]
Oct 30 22:14:20.888986 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:14:20.888954    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d8bd465_4a7c_4480_9079_eac8aed7c7a2.slice/crio-733aafcfe807959f771314c686de588cb52ed02c0c46c59ff1380a7f039729a0 WatchSource:0}: Error finding container 733aafcfe807959f771314c686de588cb52ed02c0c46c59ff1380a7f039729a0: Status 404 returned error can't find the container with id 733aafcfe807959f771314c686de588cb52ed02c0c46c59ff1380a7f039729a0
Oct 30 22:14:21.036184 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:21.036143    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:21.150313 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:21.149814    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a4257d-c6b8d878d-kr4t7" event={"ID":"7d8bd465-4a7c-4480-9079-eac8aed7c7a2","Type":"ContainerStarted","Data":"733aafcfe807959f771314c686de588cb52ed02c0c46c59ff1380a7f039729a0"}
Oct 30 22:14:23.987142 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:23.987094    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:24.189469 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:24.188819    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:24.221798 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:24.221750    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a4257d-c6b8d878d-kr4t7" event={"ID":"7d8bd465-4a7c-4480-9079-eac8aed7c7a2","Type":"ContainerStarted","Data":"7cbc91cf4acb3c900deedec0372df35742ee58e3904f2f4d0615c2112e7fa2a7"}
Oct 30 22:14:24.222019 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:24.221880    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-a4257d-c6b8d878d-kr4t7"
Oct 30 22:14:24.370879 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:24.370817    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-a4257d-c6b8d878d-kr4t7" podStartSLOduration=1.869612904 podStartE2EDuration="4.370800347s" totalImagesPullingTime="2.501187443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:14:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:14:24.333739203 +0000 UTC m=+2206.511227821" watchObservedRunningTime="2026-10-30 22:14:24.370800347 +0000 UTC m=+2206.548288964"
Oct 30 22:14:25.745400 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:25.745359    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274"]
Oct 30 22:14:25.750477 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:25.750448    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274"
Oct 30 22:14:25.800977 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:25.800940    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-c4bfac-serving-cert\""
Oct 30 22:14:25.851595 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:25.851555    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqcsl\" (UniqueName: \"kubernetes.io/projected/16ac9cf9-cfd8-42dd-9a72-094ff81da9b2-kube-api-access-gqcsl\") pod \"maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274\" (UID: \"16ac9cf9-cfd8-42dd-9a72-094ff81da9b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274"
Oct 30 22:14:25.851780 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:25.851628    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16ac9cf9-cfd8-42dd-9a72-094ff81da9b2-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274\" (UID: \"16ac9cf9-cfd8-42dd-9a72-094ff81da9b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274"
Oct 30 22:14:25.851780 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:25.851731    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16ac9cf9-cfd8-42dd-9a72-094ff81da9b2-maas-api-tls\") pod \"maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274\" (UID: \"16ac9cf9-cfd8-42dd-9a72-094ff81da9b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274"
Oct 30 22:14:25.851914 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:25.851829    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16ac9cf9-cfd8-42dd-9a72-094ff81da9b2-metrics-certs\") pod \"maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274\" (UID: \"16ac9cf9-cfd8-42dd-9a72-094ff81da9b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274"
Oct 30 22:14:25.852172 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:25.852153    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274"]
Oct 30 22:14:25.953132 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:25.953091    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqcsl\" (UniqueName: \"kubernetes.io/projected/16ac9cf9-cfd8-42dd-9a72-094ff81da9b2-kube-api-access-gqcsl\") pod \"maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274\" (UID: \"16ac9cf9-cfd8-42dd-9a72-094ff81da9b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274"
Oct 30 22:14:25.953367 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:25.953189    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16ac9cf9-cfd8-42dd-9a72-094ff81da9b2-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274\" (UID: \"16ac9cf9-cfd8-42dd-9a72-094ff81da9b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274"
Oct 30 22:14:25.953367 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:25.953221    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16ac9cf9-cfd8-42dd-9a72-094ff81da9b2-maas-api-tls\") pod \"maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274\" (UID: \"16ac9cf9-cfd8-42dd-9a72-094ff81da9b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274"
Oct 30 22:14:25.953367 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:25.953327    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16ac9cf9-cfd8-42dd-9a72-094ff81da9b2-metrics-certs\") pod \"maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274\" (UID: \"16ac9cf9-cfd8-42dd-9a72-094ff81da9b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274"
Oct 30 22:14:25.953572 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:14:25.953409    4871 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4bfac-serving-cert: secret "maas-api-e2e-ipp-b-c4bfac-serving-cert" not found
Oct 30 22:14:25.953572 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:14:25.953516    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16ac9cf9-cfd8-42dd-9a72-094ff81da9b2-maas-api-tls podName:16ac9cf9-cfd8-42dd-9a72-094ff81da9b2 nodeName:}" failed. No retries permitted until 2026-10-30 22:14:26.453482699 +0000 UTC m=+2208.630971313 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/16ac9cf9-cfd8-42dd-9a72-094ff81da9b2-maas-api-tls") pod "maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274" (UID: "16ac9cf9-cfd8-42dd-9a72-094ff81da9b2") : secret "maas-api-e2e-ipp-b-c4bfac-serving-cert" not found
Oct 30 22:14:25.953990 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:25.953969    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16ac9cf9-cfd8-42dd-9a72-094ff81da9b2-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274\" (UID: \"16ac9cf9-cfd8-42dd-9a72-094ff81da9b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274"
Oct 30 22:14:25.955908 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:25.955886    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16ac9cf9-cfd8-42dd-9a72-094ff81da9b2-metrics-certs\") pod \"maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274\" (UID: \"16ac9cf9-cfd8-42dd-9a72-094ff81da9b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274"
Oct 30 22:14:25.963551 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:25.963510    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:26.019312 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:26.018709    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqcsl\" (UniqueName: \"kubernetes.io/projected/16ac9cf9-cfd8-42dd-9a72-094ff81da9b2-kube-api-access-gqcsl\") pod \"maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274\" (UID: \"16ac9cf9-cfd8-42dd-9a72-094ff81da9b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274"
Oct 30 22:14:26.132046 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:26.132002    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:26.312403 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:26.312307    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:26.461037 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:26.460996    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16ac9cf9-cfd8-42dd-9a72-094ff81da9b2-maas-api-tls\") pod \"maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274\" (UID: \"16ac9cf9-cfd8-42dd-9a72-094ff81da9b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274"
Oct 30 22:14:26.463737 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:26.463704    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16ac9cf9-cfd8-42dd-9a72-094ff81da9b2-maas-api-tls\") pod \"maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274\" (UID: \"16ac9cf9-cfd8-42dd-9a72-094ff81da9b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274"
Oct 30 22:14:26.665524 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:26.665486    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274"
Oct 30 22:14:27.048734 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:14:27.048692    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16ac9cf9_cfd8_42dd_9a72_094ff81da9b2.slice/crio-9639fa0288e46ccdc968a72d60eb89d88562d83d59e38a51d1d3db4c8a9c9599 WatchSource:0}: Error finding container 9639fa0288e46ccdc968a72d60eb89d88562d83d59e38a51d1d3db4c8a9c9599: Status 404 returned error can't find the container with id 9639fa0288e46ccdc968a72d60eb89d88562d83d59e38a51d1d3db4c8a9c9599
Oct 30 22:14:27.189032 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:27.188994    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274"]
Oct 30 22:14:27.257634 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:27.257599    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274" event={"ID":"16ac9cf9-cfd8-42dd-9a72-094ff81da9b2","Type":"ContainerStarted","Data":"9639fa0288e46ccdc968a72d60eb89d88562d83d59e38a51d1d3db4c8a9c9599"}
Oct 30 22:14:28.267136 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:28.267097    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274" event={"ID":"16ac9cf9-cfd8-42dd-9a72-094ff81da9b2","Type":"ContainerStarted","Data":"e8d016837e1d5cc59474a114f42449a09a528a07ccccaa649826d39b3d9ddd53"}
Oct 30 22:14:28.267582 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:28.267198    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274"
Oct 30 22:14:28.475484 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:28.475425    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274" podStartSLOduration=2.463119551 podStartE2EDuration="3.475411032s" totalImagesPullingTime="1.012291481s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:14:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:14:28.42494164 +0000 UTC m=+2210.602430258" watchObservedRunningTime="2026-10-30 22:14:28.475411032 +0000 UTC m=+2210.652899649"
Oct 30 22:14:30.958926 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:30.958871    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:34.282089 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:34.282058    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274"
Oct 30 22:14:34.334755 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:34.334714    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:34.836559 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:34.836523    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:35.238260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:35.237428    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-a4257d-c6b8d878d-kr4t7"
Oct 30 22:14:35.569857 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:14:35.569773    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:14:37.062358 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:37.062323    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4"]
Oct 30 22:14:37.066781 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:37.066758    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4"
Oct 30 22:14:37.117454 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:37.117413    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-5d2b13-serving-cert\""
Oct 30 22:14:37.160191 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:37.160158    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe4334e5-e953-4fdb-901d-1350c70012f1-maas-api-tls\") pod \"maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4\" (UID: \"fe4334e5-e953-4fdb-901d-1350c70012f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4"
Oct 30 22:14:37.160384 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:37.160310    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe4334e5-e953-4fdb-901d-1350c70012f1-metrics-certs\") pod \"maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4\" (UID: \"fe4334e5-e953-4fdb-901d-1350c70012f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4"
Oct 30 22:14:37.160384 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:37.160369    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe4334e5-e953-4fdb-901d-1350c70012f1-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4\" (UID: \"fe4334e5-e953-4fdb-901d-1350c70012f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4"
Oct 30 22:14:37.160481 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:37.160394    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtwql\" (UniqueName: \"kubernetes.io/projected/fe4334e5-e953-4fdb-901d-1350c70012f1-kube-api-access-xtwql\") pod \"maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4\" (UID: \"fe4334e5-e953-4fdb-901d-1350c70012f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4"
Oct 30 22:14:37.169322 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:37.169290    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4"]
Oct 30 22:14:37.254537 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:37.254495    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:37.261960 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:37.261934    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe4334e5-e953-4fdb-901d-1350c70012f1-metrics-certs\") pod \"maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4\" (UID: \"fe4334e5-e953-4fdb-901d-1350c70012f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4"
Oct 30 22:14:37.262126 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:37.261999    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe4334e5-e953-4fdb-901d-1350c70012f1-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4\" (UID: \"fe4334e5-e953-4fdb-901d-1350c70012f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4"
Oct 30 22:14:37.262126 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:37.262021    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtwql\" (UniqueName: \"kubernetes.io/projected/fe4334e5-e953-4fdb-901d-1350c70012f1-kube-api-access-xtwql\") pod \"maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4\" (UID: \"fe4334e5-e953-4fdb-901d-1350c70012f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4"
Oct 30 22:14:37.262126 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:37.262107    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe4334e5-e953-4fdb-901d-1350c70012f1-maas-api-tls\") pod \"maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4\" (UID: \"fe4334e5-e953-4fdb-901d-1350c70012f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4"
Oct 30 22:14:37.262315 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:14:37.262221    4871 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-5d2b13-serving-cert: secret "maas-api-e2e-ipp-b-5d2b13-serving-cert" not found
Oct 30 22:14:37.262315 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:14:37.262298    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe4334e5-e953-4fdb-901d-1350c70012f1-maas-api-tls podName:fe4334e5-e953-4fdb-901d-1350c70012f1 nodeName:}" failed. No retries permitted until 2026-10-30 22:14:37.762281549 +0000 UTC m=+2219.939770157 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fe4334e5-e953-4fdb-901d-1350c70012f1-maas-api-tls") pod "maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4" (UID: "fe4334e5-e953-4fdb-901d-1350c70012f1") : secret "maas-api-e2e-ipp-b-5d2b13-serving-cert" not found
Oct 30 22:14:37.262856 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:37.262833    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe4334e5-e953-4fdb-901d-1350c70012f1-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4\" (UID: \"fe4334e5-e953-4fdb-901d-1350c70012f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4"
Oct 30 22:14:37.264657 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:37.264633    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe4334e5-e953-4fdb-901d-1350c70012f1-metrics-certs\") pod \"maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4\" (UID: \"fe4334e5-e953-4fdb-901d-1350c70012f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4"
Oct 30 22:14:37.323010 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:37.322928    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtwql\" (UniqueName: \"kubernetes.io/projected/fe4334e5-e953-4fdb-901d-1350c70012f1-kube-api-access-xtwql\") pod \"maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4\" (UID: \"fe4334e5-e953-4fdb-901d-1350c70012f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4"
Oct 30 22:14:37.409991 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:37.409958    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:37.616874 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:37.616834    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:37.771309 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:37.771269    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe4334e5-e953-4fdb-901d-1350c70012f1-maas-api-tls\") pod \"maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4\" (UID: \"fe4334e5-e953-4fdb-901d-1350c70012f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4"
Oct 30 22:14:37.773733 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:37.773702    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe4334e5-e953-4fdb-901d-1350c70012f1-maas-api-tls\") pod \"maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4\" (UID: \"fe4334e5-e953-4fdb-901d-1350c70012f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4"
Oct 30 22:14:37.981603 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:37.981518    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4"
Oct 30 22:14:38.390843 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:38.390093    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4"]
Oct 30 22:14:38.394664 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:14:38.394629    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe4334e5_e953_4fdb_901d_1350c70012f1.slice/crio-c8cc7a52890392b71536c0b07cc7381753333d56c95092560f35a73d94319e0e WatchSource:0}: Error finding container c8cc7a52890392b71536c0b07cc7381753333d56c95092560f35a73d94319e0e: Status 404 returned error can't find the container with id c8cc7a52890392b71536c0b07cc7381753333d56c95092560f35a73d94319e0e
Oct 30 22:14:38.413492 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:38.413459    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4" event={"ID":"fe4334e5-e953-4fdb-901d-1350c70012f1","Type":"ContainerStarted","Data":"c8cc7a52890392b71536c0b07cc7381753333d56c95092560f35a73d94319e0e"}
Oct 30 22:14:39.424919 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:39.424872    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4" event={"ID":"fe4334e5-e953-4fdb-901d-1350c70012f1","Type":"ContainerStarted","Data":"1a6dc0a4e3a5c6425556f270a880a19ff7125e9aaf83e34c88cba168d3ecfb9e"}
Oct 30 22:14:39.425349 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:39.424989    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4"
Oct 30 22:14:39.587946 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:39.587760    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4" podStartSLOduration=1.681401364 podStartE2EDuration="2.587744556s" totalImagesPullingTime="906.343192ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:14:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:14:39.540492987 +0000 UTC m=+2221.717981605" watchObservedRunningTime="2026-10-30 22:14:39.587744556 +0000 UTC m=+2221.765233172"
Oct 30 22:14:41.298025 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:41.297992    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:42.187353 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:42.187320    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:45.438609 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:45.438578    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4"
Oct 30 22:14:46.846172 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:46.846111    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:51.886263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:51.886204    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:58.041332 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:58.041267    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:58.204450 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:14:58.204413    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:00.337899 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:00.337860    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889975-6bhfx"]
Oct 30 22:15:00.343008 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:00.342984    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889975-6bhfx"
Oct 30 22:15:00.363425 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:00.363390    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f0b8c9-29889975-bv2s9"]
Oct 30 22:15:00.367628 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:00.367605    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f0b8c9-29889975-bv2s9"
Oct 30 22:15:00.403925 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:00.403892    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mlvtf\""
Oct 30 22:15:00.448836 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:00.448803    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889975-6bhfx"]
Oct 30 22:15:00.454294 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:00.454265    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f0b8c9-29889975-bv2s9"]
Oct 30 22:15:00.494297 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:00.494261    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1ff7ed9-fae1-4646-948e-aa1d82508cb0-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-ipp-a-f0b8c9-29889975-bv2s9\" (UID: \"e1ff7ed9-fae1-4646-948e-aa1d82508cb0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f0b8c9-29889975-bv2s9"
Oct 30 22:15:00.494439 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:00.494412    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrbr9\" (UniqueName: \"kubernetes.io/projected/29cba590-3fb8-4d7c-a357-aebb92f595cb-kube-api-access-wrbr9\") pod \"maas-api-key-cleanup-e2e-worker-w4-26743a-29889975-6bhfx\" (UID: \"29cba590-3fb8-4d7c-a357-aebb92f595cb\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889975-6bhfx"
Oct 30 22:15:00.494499 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:00.494453    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8gkw\" (UniqueName: \"kubernetes.io/projected/e1ff7ed9-fae1-4646-948e-aa1d82508cb0-kube-api-access-v8gkw\") pod \"maas-api-key-cleanup-e2e-ipp-a-f0b8c9-29889975-bv2s9\" (UID: \"e1ff7ed9-fae1-4646-948e-aa1d82508cb0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f0b8c9-29889975-bv2s9"
Oct 30 22:15:00.494499 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:00.494473    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29cba590-3fb8-4d7c-a357-aebb92f595cb-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-26743a-29889975-6bhfx\" (UID: \"29cba590-3fb8-4d7c-a357-aebb92f595cb\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889975-6bhfx"
Oct 30 22:15:00.595901 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:00.595821    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1ff7ed9-fae1-4646-948e-aa1d82508cb0-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-ipp-a-f0b8c9-29889975-bv2s9\" (UID: \"e1ff7ed9-fae1-4646-948e-aa1d82508cb0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f0b8c9-29889975-bv2s9"
Oct 30 22:15:00.596050 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:00.595918    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrbr9\" (UniqueName: \"kubernetes.io/projected/29cba590-3fb8-4d7c-a357-aebb92f595cb-kube-api-access-wrbr9\") pod \"maas-api-key-cleanup-e2e-worker-w4-26743a-29889975-6bhfx\" (UID: \"29cba590-3fb8-4d7c-a357-aebb92f595cb\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889975-6bhfx"
Oct 30 22:15:00.596050 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:00.595948    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8gkw\" (UniqueName: \"kubernetes.io/projected/e1ff7ed9-fae1-4646-948e-aa1d82508cb0-kube-api-access-v8gkw\") pod \"maas-api-key-cleanup-e2e-ipp-a-f0b8c9-29889975-bv2s9\" (UID: \"e1ff7ed9-fae1-4646-948e-aa1d82508cb0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f0b8c9-29889975-bv2s9"
Oct 30 22:15:00.596050 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:00.595966    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29cba590-3fb8-4d7c-a357-aebb92f595cb-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-26743a-29889975-6bhfx\" (UID: \"29cba590-3fb8-4d7c-a357-aebb92f595cb\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889975-6bhfx"
Oct 30 22:15:00.596619 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:00.596590    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1ff7ed9-fae1-4646-948e-aa1d82508cb0-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-ipp-a-f0b8c9-29889975-bv2s9\" (UID: \"e1ff7ed9-fae1-4646-948e-aa1d82508cb0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f0b8c9-29889975-bv2s9"
Oct 30 22:15:00.596767 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:00.596750    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29cba590-3fb8-4d7c-a357-aebb92f595cb-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-26743a-29889975-6bhfx\" (UID: \"29cba590-3fb8-4d7c-a357-aebb92f595cb\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889975-6bhfx"
Oct 30 22:15:00.664936 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:00.664913    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrbr9\" (UniqueName: \"kubernetes.io/projected/29cba590-3fb8-4d7c-a357-aebb92f595cb-kube-api-access-wrbr9\") pod \"maas-api-key-cleanup-e2e-worker-w4-26743a-29889975-6bhfx\" (UID: \"29cba590-3fb8-4d7c-a357-aebb92f595cb\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889975-6bhfx"
Oct 30 22:15:00.670045 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:00.670024    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8gkw\" (UniqueName: \"kubernetes.io/projected/e1ff7ed9-fae1-4646-948e-aa1d82508cb0-kube-api-access-v8gkw\") pod \"maas-api-key-cleanup-e2e-ipp-a-f0b8c9-29889975-bv2s9\" (UID: \"e1ff7ed9-fae1-4646-948e-aa1d82508cb0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f0b8c9-29889975-bv2s9"
Oct 30 22:15:00.671877 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:00.671861    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889975-6bhfx"
Oct 30 22:15:00.681417 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:00.681395    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f0b8c9-29889975-bv2s9"
Oct 30 22:15:01.083136 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:15:01.083100    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29cba590_3fb8_4d7c_a357_aebb92f595cb.slice/crio-db962aad2cc14ca0f3057cc1803bcf3fcb133c0a079b3e1307dd48d59e2b6767 WatchSource:0}: Error finding container db962aad2cc14ca0f3057cc1803bcf3fcb133c0a079b3e1307dd48d59e2b6767: Status 404 returned error can't find the container with id db962aad2cc14ca0f3057cc1803bcf3fcb133c0a079b3e1307dd48d59e2b6767
Oct 30 22:15:01.083312 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:01.083253    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889975-6bhfx"]
Oct 30 22:15:01.118208 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:01.118182    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f0b8c9-29889975-bv2s9"]
Oct 30 22:15:01.118856 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:15:01.118830    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1ff7ed9_fae1_4646_948e_aa1d82508cb0.slice/crio-9eb5281a100aa8c5e15573920bdf7014690ac676272af2f9ff2b0e1222334d7a WatchSource:0}: Error finding container 9eb5281a100aa8c5e15573920bdf7014690ac676272af2f9ff2b0e1222334d7a: Status 404 returned error can't find the container with id 9eb5281a100aa8c5e15573920bdf7014690ac676272af2f9ff2b0e1222334d7a
Oct 30 22:15:01.589832 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:01.589790    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889975-6bhfx" event={"ID":"29cba590-3fb8-4d7c-a357-aebb92f595cb","Type":"ContainerStarted","Data":"e680b9876cf39201d22bea5bd3588af4907feb01dd794459ca8e3f4c8db09ac2"}
Oct 30 22:15:01.590342 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:01.589859    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889975-6bhfx" event={"ID":"29cba590-3fb8-4d7c-a357-aebb92f595cb","Type":"ContainerStarted","Data":"db962aad2cc14ca0f3057cc1803bcf3fcb133c0a079b3e1307dd48d59e2b6767"}
Oct 30 22:15:01.591575 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:01.591543    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f0b8c9-29889975-bv2s9" event={"ID":"e1ff7ed9-fae1-4646-948e-aa1d82508cb0","Type":"ContainerStarted","Data":"90af5012c95e0f0e2214395ed0b0e67827901410d2c6fed793fc5a30d4121126"}
Oct 30 22:15:01.591691 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:01.591602    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f0b8c9-29889975-bv2s9" event={"ID":"e1ff7ed9-fae1-4646-948e-aa1d82508cb0","Type":"ContainerStarted","Data":"9eb5281a100aa8c5e15573920bdf7014690ac676272af2f9ff2b0e1222334d7a"}
Oct 30 22:15:01.728807 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:01.728675    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889975-6bhfx" podStartSLOduration=1.728655151 podStartE2EDuration="1.728655151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:15:01.723933568 +0000 UTC m=+2243.901422186" watchObservedRunningTime="2026-10-30 22:15:01.728655151 +0000 UTC m=+2243.906143769"
Oct 30 22:15:01.848375 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:01.848313    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f0b8c9-29889975-bv2s9" podStartSLOduration=1.8482920840000001 podStartE2EDuration="1.848292084s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:15:01.841322965 +0000 UTC m=+2244.018811581" watchObservedRunningTime="2026-10-30 22:15:01.848292084 +0000 UTC m=+2244.025780702"
Oct 30 22:15:02.601003 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:02.600970    4871 generic.go:356] "Generic (PLEG): container finished" podID="e1ff7ed9-fae1-4646-948e-aa1d82508cb0" containerID="90af5012c95e0f0e2214395ed0b0e67827901410d2c6fed793fc5a30d4121126" exitCode=60
Oct 30 22:15:02.601494 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:02.601052    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f0b8c9-29889975-bv2s9" event={"ID":"e1ff7ed9-fae1-4646-948e-aa1d82508cb0","Type":"ContainerDied","Data":"90af5012c95e0f0e2214395ed0b0e67827901410d2c6fed793fc5a30d4121126"}
Oct 30 22:15:02.601494 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:02.601356    4871 scope.go:122] "RemoveContainer" containerID="90af5012c95e0f0e2214395ed0b0e67827901410d2c6fed793fc5a30d4121126"
Oct 30 22:15:02.603194 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:02.603172    4871 generic.go:356] "Generic (PLEG): container finished" podID="29cba590-3fb8-4d7c-a357-aebb92f595cb" containerID="e680b9876cf39201d22bea5bd3588af4907feb01dd794459ca8e3f4c8db09ac2" exitCode=60
Oct 30 22:15:02.603314 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:02.603290    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889975-6bhfx" event={"ID":"29cba590-3fb8-4d7c-a357-aebb92f595cb","Type":"ContainerDied","Data":"e680b9876cf39201d22bea5bd3588af4907feb01dd794459ca8e3f4c8db09ac2"}
Oct 30 22:15:02.603549 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:02.603531    4871 scope.go:122] "RemoveContainer" containerID="e680b9876cf39201d22bea5bd3588af4907feb01dd794459ca8e3f4c8db09ac2"
Oct 30 22:15:03.234017 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:03.233794    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:03.618002 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:03.617966    4871 generic.go:356] "Generic (PLEG): container finished" podID="29cba590-3fb8-4d7c-a357-aebb92f595cb" containerID="169651562fd0bf4b6c7361a73f370343731638d3e2f362deb5c192ea7a56b68c" exitCode=60
Oct 30 22:15:03.618489 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:03.618162    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889975-6bhfx" event={"ID":"29cba590-3fb8-4d7c-a357-aebb92f595cb","Type":"ContainerDied","Data":"169651562fd0bf4b6c7361a73f370343731638d3e2f362deb5c192ea7a56b68c"}
Oct 30 22:15:03.618489 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:03.618281    4871 scope.go:122] "RemoveContainer" containerID="e680b9876cf39201d22bea5bd3588af4907feb01dd794459ca8e3f4c8db09ac2"
Oct 30 22:15:03.618489 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:03.618432    4871 scope.go:122] "RemoveContainer" containerID="169651562fd0bf4b6c7361a73f370343731638d3e2f362deb5c192ea7a56b68c"
Oct 30 22:15:03.618785 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:15:03.618762    4871 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-26743a-29889975-6bhfx_odh-ai-gateway-infra(29cba590-3fb8-4d7c-a357-aebb92f595cb)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889975-6bhfx" podUID="29cba590-3fb8-4d7c-a357-aebb92f595cb"
Oct 30 22:15:03.620892 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:03.620867    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f0b8c9-29889975-bv2s9" event={"ID":"e1ff7ed9-fae1-4646-948e-aa1d82508cb0","Type":"ContainerStarted","Data":"e5e6a7a0ff88e19893c9f1fd3e01b828b1d2607aa72760febb2b1158ef70bc53"}
Oct 30 22:15:03.621269 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:03.621221    4871 scope.go:122] "RemoveContainer" containerID="e5e6a7a0ff88e19893c9f1fd3e01b828b1d2607aa72760febb2b1158ef70bc53"
Oct 30 22:15:03.621540 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:15:03.621522    4871 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ipp-a-f0b8c9-29889975-bv2s9_odh-ai-gateway-infra(e1ff7ed9-fae1-4646-948e-aa1d82508cb0)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f0b8c9-29889975-bv2s9" podUID="e1ff7ed9-fae1-4646-948e-aa1d82508cb0"
Oct 30 22:15:04.634506 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:04.634469    4871 scope.go:122] "RemoveContainer" containerID="169651562fd0bf4b6c7361a73f370343731638d3e2f362deb5c192ea7a56b68c"
Oct 30 22:15:04.634985 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:15:04.634873    4871 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-26743a-29889975-6bhfx_odh-ai-gateway-infra(29cba590-3fb8-4d7c-a357-aebb92f595cb)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889975-6bhfx" podUID="29cba590-3fb8-4d7c-a357-aebb92f595cb"
Oct 30 22:15:04.636366 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:04.636337    4871 generic.go:356] "Generic (PLEG): container finished" podID="e1ff7ed9-fae1-4646-948e-aa1d82508cb0" containerID="e5e6a7a0ff88e19893c9f1fd3e01b828b1d2607aa72760febb2b1158ef70bc53" exitCode=60
Oct 30 22:15:04.636494 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:04.636422    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f0b8c9-29889975-bv2s9" event={"ID":"e1ff7ed9-fae1-4646-948e-aa1d82508cb0","Type":"ContainerDied","Data":"e5e6a7a0ff88e19893c9f1fd3e01b828b1d2607aa72760febb2b1158ef70bc53"}
Oct 30 22:15:04.636494 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:04.636482    4871 scope.go:122] "RemoveContainer" containerID="90af5012c95e0f0e2214395ed0b0e67827901410d2c6fed793fc5a30d4121126"
Oct 30 22:15:04.636800 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:04.636777    4871 scope.go:122] "RemoveContainer" containerID="e5e6a7a0ff88e19893c9f1fd3e01b828b1d2607aa72760febb2b1158ef70bc53"
Oct 30 22:15:04.637162 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:15:04.637133    4871 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ipp-a-f0b8c9-29889975-bv2s9_odh-ai-gateway-infra(e1ff7ed9-fae1-4646-948e-aa1d82508cb0)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f0b8c9-29889975-bv2s9" podUID="e1ff7ed9-fae1-4646-948e-aa1d82508cb0"
Oct 30 22:15:05.568153 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:15:05.568120    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:15:05.648860 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:05.648833    4871 scope.go:122] "RemoveContainer" containerID="e5e6a7a0ff88e19893c9f1fd3e01b828b1d2607aa72760febb2b1158ef70bc53"
Oct 30 22:15:05.649276 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:15:05.649208    4871 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ipp-a-f0b8c9-29889975-bv2s9_odh-ai-gateway-infra(e1ff7ed9-fae1-4646-948e-aa1d82508cb0)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f0b8c9-29889975-bv2s9" podUID="e1ff7ed9-fae1-4646-948e-aa1d82508cb0"
Oct 30 22:15:08.222946 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:08.222897    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:13.040610 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:13.040576    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:14.953154 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:14.953113    4871 scope.go:122] "RemoveContainer" containerID="169651562fd0bf4b6c7361a73f370343731638d3e2f362deb5c192ea7a56b68c"
Oct 30 22:15:15.779387 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:15.779347    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889975-6bhfx" event={"ID":"29cba590-3fb8-4d7c-a357-aebb92f595cb","Type":"ContainerStarted","Data":"d9d6caae94d7996d107830da7e85f68a0d83714bbbaa780b7961bbb17b63eeaf"}
Oct 30 22:15:15.953775 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:15.953740    4871 scope.go:122] "RemoveContainer" containerID="e5e6a7a0ff88e19893c9f1fd3e01b828b1d2607aa72760febb2b1158ef70bc53"
Oct 30 22:15:16.788832 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:16.788792    4871 generic.go:356] "Generic (PLEG): container finished" podID="29cba590-3fb8-4d7c-a357-aebb92f595cb" containerID="d9d6caae94d7996d107830da7e85f68a0d83714bbbaa780b7961bbb17b63eeaf" exitCode=60
Oct 30 22:15:16.789005 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:16.788857    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889975-6bhfx" event={"ID":"29cba590-3fb8-4d7c-a357-aebb92f595cb","Type":"ContainerDied","Data":"d9d6caae94d7996d107830da7e85f68a0d83714bbbaa780b7961bbb17b63eeaf"}
Oct 30 22:15:16.789005 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:16.788916    4871 scope.go:122] "RemoveContainer" containerID="169651562fd0bf4b6c7361a73f370343731638d3e2f362deb5c192ea7a56b68c"
Oct 30 22:15:16.789349 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:16.789327    4871 scope.go:122] "RemoveContainer" containerID="d9d6caae94d7996d107830da7e85f68a0d83714bbbaa780b7961bbb17b63eeaf"
Oct 30 22:15:16.789705 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:15:16.789680    4871 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-26743a-29889975-6bhfx_odh-ai-gateway-infra(29cba590-3fb8-4d7c-a357-aebb92f595cb)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889975-6bhfx" podUID="29cba590-3fb8-4d7c-a357-aebb92f595cb"
Oct 30 22:15:16.791441 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:16.791416    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f0b8c9-29889975-bv2s9" event={"ID":"e1ff7ed9-fae1-4646-948e-aa1d82508cb0","Type":"ContainerStarted","Data":"b898bf67126b8c580adbc206d04f6a0d481da825c865a93ad48163586d81ab10"}
Oct 30 22:15:16.909384 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:16.909335    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889975-6bhfx"]
Oct 30 22:15:17.813767 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:17.813735    4871 generic.go:356] "Generic (PLEG): container finished" podID="6674b9d4-017b-4109-9262-bc8ad1244712" containerID="7b93e238cb14ccef31ed03ca1a25e2ede53c102435d6fb5fe8b45a2c87af535b" exitCode=0
Oct 30 22:15:17.814224 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:17.813804    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856" event={"ID":"6674b9d4-017b-4109-9262-bc8ad1244712","Type":"ContainerDied","Data":"7b93e238cb14ccef31ed03ca1a25e2ede53c102435d6fb5fe8b45a2c87af535b"}
Oct 30 22:15:17.814224 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:17.814220    4871 scope.go:122] "RemoveContainer" containerID="7b93e238cb14ccef31ed03ca1a25e2ede53c102435d6fb5fe8b45a2c87af535b"
Oct 30 22:15:17.815877 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:17.815848    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:15:17.815980 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:17.815876    4871 generic.go:356] "Generic (PLEG): container finished" podID="e1ff7ed9-fae1-4646-948e-aa1d82508cb0" containerID="b898bf67126b8c580adbc206d04f6a0d481da825c865a93ad48163586d81ab10" exitCode=60
Oct 30 22:15:17.815980 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:17.815912    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:15:17.815980 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:17.815924    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f0b8c9-29889975-bv2s9" event={"ID":"e1ff7ed9-fae1-4646-948e-aa1d82508cb0","Type":"ContainerDied","Data":"b898bf67126b8c580adbc206d04f6a0d481da825c865a93ad48163586d81ab10"}
Oct 30 22:15:17.815980 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:17.815972    4871 scope.go:122] "RemoveContainer" containerID="e5e6a7a0ff88e19893c9f1fd3e01b828b1d2607aa72760febb2b1158ef70bc53"
Oct 30 22:15:17.815980 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:17.815972    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:15:17.816298 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:17.816277    4871 scope.go:122] "RemoveContainer" containerID="b898bf67126b8c580adbc206d04f6a0d481da825c865a93ad48163586d81ab10"
Oct 30 22:15:17.816664 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:15:17.816531    4871 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ipp-a-f0b8c9-29889975-bv2s9_odh-ai-gateway-infra(e1ff7ed9-fae1-4646-948e-aa1d82508cb0)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f0b8c9-29889975-bv2s9" podUID="e1ff7ed9-fae1-4646-948e-aa1d82508cb0"
Oct 30 22:15:18.000644 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:18.000614    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889975-6bhfx"
Oct 30 22:15:18.049656 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:18.049624    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f0b8c9-29889975-bv2s9"]
Oct 30 22:15:18.071055 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:18.070968    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/29cba590-3fb8-4d7c-a357-aebb92f595cb-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29cba590-3fb8-4d7c-a357-aebb92f595cb-maas-api-service-ca\") pod \"29cba590-3fb8-4d7c-a357-aebb92f595cb\" (UID: \"29cba590-3fb8-4d7c-a357-aebb92f595cb\") "
Oct 30 22:15:18.071222 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:18.071202    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/29cba590-3fb8-4d7c-a357-aebb92f595cb-kube-api-access-wrbr9\" (UniqueName: \"kubernetes.io/projected/29cba590-3fb8-4d7c-a357-aebb92f595cb-kube-api-access-wrbr9\") pod \"29cba590-3fb8-4d7c-a357-aebb92f595cb\" (UID: \"29cba590-3fb8-4d7c-a357-aebb92f595cb\") "
Oct 30 22:15:18.071428 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:18.071405    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29cba590-3fb8-4d7c-a357-aebb92f595cb-maas-api-service-ca" pod "29cba590-3fb8-4d7c-a357-aebb92f595cb" (UID: "29cba590-3fb8-4d7c-a357-aebb92f595cb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:15:18.071937 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:18.071874    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29cba590-3fb8-4d7c-a357-aebb92f595cb-maas-api-service-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:15:18.073457 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:18.073422    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29cba590-3fb8-4d7c-a357-aebb92f595cb-kube-api-access-wrbr9" pod "29cba590-3fb8-4d7c-a357-aebb92f595cb" (UID: "29cba590-3fb8-4d7c-a357-aebb92f595cb"). InnerVolumeSpecName "kube-api-access-wrbr9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:18.172728 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:18.172692    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrbr9\" (UniqueName: \"kubernetes.io/projected/29cba590-3fb8-4d7c-a357-aebb92f595cb-kube-api-access-wrbr9\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:15:18.825075 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:18.825035    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889975-6bhfx"
Oct 30 22:15:18.825613 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:18.825078    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889975-6bhfx" event={"ID":"29cba590-3fb8-4d7c-a357-aebb92f595cb","Type":"ContainerDied","Data":"db962aad2cc14ca0f3057cc1803bcf3fcb133c0a079b3e1307dd48d59e2b6767"}
Oct 30 22:15:18.825613 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:18.825139    4871 scope.go:122] "RemoveContainer" containerID="d9d6caae94d7996d107830da7e85f68a0d83714bbbaa780b7961bbb17b63eeaf"
Oct 30 22:15:18.827651 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:18.827622    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856" event={"ID":"6674b9d4-017b-4109-9262-bc8ad1244712","Type":"ContainerStarted","Data":"da178ee02a1013c7b9cfc72cd53ca60dbe18900ff5139ea376e4466cc1b24469"}
Oct 30 22:15:18.999525 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:18.999491    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f0b8c9-29889975-bv2s9"
Oct 30 22:15:19.085010 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.084914    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e1ff7ed9-fae1-4646-948e-aa1d82508cb0-kube-api-access-v8gkw\" (UniqueName: \"kubernetes.io/projected/e1ff7ed9-fae1-4646-948e-aa1d82508cb0-kube-api-access-v8gkw\") pod \"e1ff7ed9-fae1-4646-948e-aa1d82508cb0\" (UID: \"e1ff7ed9-fae1-4646-948e-aa1d82508cb0\") "
Oct 30 22:15:19.085010 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.084991    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e1ff7ed9-fae1-4646-948e-aa1d82508cb0-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1ff7ed9-fae1-4646-948e-aa1d82508cb0-maas-api-service-ca\") pod \"e1ff7ed9-fae1-4646-948e-aa1d82508cb0\" (UID: \"e1ff7ed9-fae1-4646-948e-aa1d82508cb0\") "
Oct 30 22:15:19.085379 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.085353    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1ff7ed9-fae1-4646-948e-aa1d82508cb0-maas-api-service-ca" pod "e1ff7ed9-fae1-4646-948e-aa1d82508cb0" (UID: "e1ff7ed9-fae1-4646-948e-aa1d82508cb0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:15:19.085772 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.085747    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1ff7ed9-fae1-4646-948e-aa1d82508cb0-maas-api-service-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:15:19.087200 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.087175    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1ff7ed9-fae1-4646-948e-aa1d82508cb0-kube-api-access-v8gkw" pod "e1ff7ed9-fae1-4646-948e-aa1d82508cb0" (UID: "e1ff7ed9-fae1-4646-948e-aa1d82508cb0"). InnerVolumeSpecName "kube-api-access-v8gkw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:19.091345 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.091321    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p"]
Oct 30 22:15:19.091989 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.091974    4871 cpu_manager.go:414] "containerMap: removing container" podUID="e1ff7ed9-fae1-4646-948e-aa1d82508cb0" containerName="cleanup"
Oct 30 22:15:19.092057 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.091991    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e1ff7ed9-fae1-4646-948e-aa1d82508cb0" containerName="cleanup"
Oct 30 22:15:19.092057 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.092036    4871 cpu_manager.go:414] "containerMap: removing container" podUID="e1ff7ed9-fae1-4646-948e-aa1d82508cb0" containerName="cleanup"
Oct 30 22:15:19.092057 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.092043    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e1ff7ed9-fae1-4646-948e-aa1d82508cb0" containerName="cleanup"
Oct 30 22:15:19.092057 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.092053    4871 cpu_manager.go:414] "containerMap: removing container" podUID="29cba590-3fb8-4d7c-a357-aebb92f595cb" containerName="cleanup"
Oct 30 22:15:19.092057 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.092058    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="29cba590-3fb8-4d7c-a357-aebb92f595cb" containerName="cleanup"
Oct 30 22:15:19.092216 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.092079    4871 cpu_manager.go:414] "containerMap: removing container" podUID="e1ff7ed9-fae1-4646-948e-aa1d82508cb0" containerName="cleanup"
Oct 30 22:15:19.092216 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.092086    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e1ff7ed9-fae1-4646-948e-aa1d82508cb0" containerName="cleanup"
Oct 30 22:15:19.092216 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.092098    4871 cpu_manager.go:414] "containerMap: removing container" podUID="29cba590-3fb8-4d7c-a357-aebb92f595cb" containerName="cleanup"
Oct 30 22:15:19.092216 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.092105    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="29cba590-3fb8-4d7c-a357-aebb92f595cb" containerName="cleanup"
Oct 30 22:15:19.092437 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.092347    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="e1ff7ed9-fae1-4646-948e-aa1d82508cb0" containerName="cleanup"
Oct 30 22:15:19.092437 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.092365    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="e1ff7ed9-fae1-4646-948e-aa1d82508cb0" containerName="cleanup"
Oct 30 22:15:19.092437 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.092376    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="29cba590-3fb8-4d7c-a357-aebb92f595cb" containerName="cleanup"
Oct 30 22:15:19.092437 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.092385    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="29cba590-3fb8-4d7c-a357-aebb92f595cb" containerName="cleanup"
Oct 30 22:15:19.092437 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.092398    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="29cba590-3fb8-4d7c-a357-aebb92f595cb" containerName="cleanup"
Oct 30 22:15:19.096051 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.096034    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p"
Oct 30 22:15:19.122938 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.122906    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889975-6bhfx"]
Oct 30 22:15:19.145809 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.145763    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-26743a-29889975-6bhfx"]
Oct 30 22:15:19.157664 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.157633    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-fe745d-serving-cert\""
Oct 30 22:15:19.161598 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.161558    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p"]
Oct 30 22:15:19.186674 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.186638    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a6d90d2-019e-41e6-9d6f-a64f537f4963-maas-api-tls\") pod \"maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p\" (UID: \"9a6d90d2-019e-41e6-9d6f-a64f537f4963\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p"
Oct 30 22:15:19.186848 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.186701    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a6d90d2-019e-41e6-9d6f-a64f537f4963-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p\" (UID: \"9a6d90d2-019e-41e6-9d6f-a64f537f4963\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p"
Oct 30 22:15:19.186848 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.186815    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a6d90d2-019e-41e6-9d6f-a64f537f4963-metrics-certs\") pod \"maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p\" (UID: \"9a6d90d2-019e-41e6-9d6f-a64f537f4963\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p"
Oct 30 22:15:19.186978 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.186963    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlb2n\" (UniqueName: \"kubernetes.io/projected/9a6d90d2-019e-41e6-9d6f-a64f537f4963-kube-api-access-wlb2n\") pod \"maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p\" (UID: \"9a6d90d2-019e-41e6-9d6f-a64f537f4963\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p"
Oct 30 22:15:19.187098 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.187087    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8gkw\" (UniqueName: \"kubernetes.io/projected/e1ff7ed9-fae1-4646-948e-aa1d82508cb0-kube-api-access-v8gkw\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:15:19.288812 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.288776    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlb2n\" (UniqueName: \"kubernetes.io/projected/9a6d90d2-019e-41e6-9d6f-a64f537f4963-kube-api-access-wlb2n\") pod \"maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p\" (UID: \"9a6d90d2-019e-41e6-9d6f-a64f537f4963\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p"
Oct 30 22:15:19.288986 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.288891    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a6d90d2-019e-41e6-9d6f-a64f537f4963-maas-api-tls\") pod \"maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p\" (UID: \"9a6d90d2-019e-41e6-9d6f-a64f537f4963\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p"
Oct 30 22:15:19.288986 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.288960    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a6d90d2-019e-41e6-9d6f-a64f537f4963-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p\" (UID: \"9a6d90d2-019e-41e6-9d6f-a64f537f4963\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p"
Oct 30 22:15:19.289125 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:15:19.289001    4871 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-fe745d-serving-cert: secret "maas-api-e2e-ipp-b-fe745d-serving-cert" not found
Oct 30 22:15:19.289125 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.289019    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a6d90d2-019e-41e6-9d6f-a64f537f4963-metrics-certs\") pod \"maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p\" (UID: \"9a6d90d2-019e-41e6-9d6f-a64f537f4963\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p"
Oct 30 22:15:19.289125 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:15:19.289078    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a6d90d2-019e-41e6-9d6f-a64f537f4963-maas-api-tls podName:9a6d90d2-019e-41e6-9d6f-a64f537f4963 nodeName:}" failed. No retries permitted until 2026-10-30 22:15:19.78905685 +0000 UTC m=+2261.966545468 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9a6d90d2-019e-41e6-9d6f-a64f537f4963-maas-api-tls") pod "maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p" (UID: "9a6d90d2-019e-41e6-9d6f-a64f537f4963") : secret "maas-api-e2e-ipp-b-fe745d-serving-cert" not found
Oct 30 22:15:19.289795 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.289768    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a6d90d2-019e-41e6-9d6f-a64f537f4963-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p\" (UID: \"9a6d90d2-019e-41e6-9d6f-a64f537f4963\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p"
Oct 30 22:15:19.291605 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.291574    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a6d90d2-019e-41e6-9d6f-a64f537f4963-metrics-certs\") pod \"maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p\" (UID: \"9a6d90d2-019e-41e6-9d6f-a64f537f4963\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p"
Oct 30 22:15:19.317342 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.317296    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"
Oct 30 22:15:19.318530 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.318500    4871 patch_prober.go:29] interesting pod/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused" start-of-body=
Oct 30 22:15:19.318674 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.318550    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856" podUID="6674b9d4-017b-4109-9262-bc8ad1244712" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused"
Oct 30 22:15:19.439442 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.428974    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlb2n\" (UniqueName: \"kubernetes.io/projected/9a6d90d2-019e-41e6-9d6f-a64f537f4963-kube-api-access-wlb2n\") pod \"maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p\" (UID: \"9a6d90d2-019e-41e6-9d6f-a64f537f4963\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p"
Oct 30 22:15:19.529288 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.529229    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:19.764360 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.764250    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:19.839262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.838621    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a6d90d2-019e-41e6-9d6f-a64f537f4963-maas-api-tls\") pod \"maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p\" (UID: \"9a6d90d2-019e-41e6-9d6f-a64f537f4963\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p"
Oct 30 22:15:19.842467 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.842431    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a6d90d2-019e-41e6-9d6f-a64f537f4963-maas-api-tls\") pod \"maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p\" (UID: \"9a6d90d2-019e-41e6-9d6f-a64f537f4963\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p"
Oct 30 22:15:19.847080 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.847055    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f0b8c9-29889975-bv2s9"
Oct 30 22:15:19.847224 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.847075    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f0b8c9-29889975-bv2s9" event={"ID":"e1ff7ed9-fae1-4646-948e-aa1d82508cb0","Type":"ContainerDied","Data":"9eb5281a100aa8c5e15573920bdf7014690ac676272af2f9ff2b0e1222334d7a"}
Oct 30 22:15:19.847224 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.847148    4871 scope.go:122] "RemoveContainer" containerID="b898bf67126b8c580adbc206d04f6a0d481da825c865a93ad48163586d81ab10"
Oct 30 22:15:19.963531 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:19.963420    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="29cba590-3fb8-4d7c-a357-aebb92f595cb" path="/var/lib/kubelet/pods/29cba590-3fb8-4d7c-a357-aebb92f595cb/volumes"
Oct 30 22:15:20.036604 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:20.036507    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p"
Oct 30 22:15:20.044286 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:20.043790    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f0b8c9-29889975-bv2s9"]
Oct 30 22:15:20.060414 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:20.060384    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"
Oct 30 22:15:20.074448 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:20.074421    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f0b8c9-29889975-bv2s9"]
Oct 30 22:15:20.336313 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:20.335734    4871 patch_prober.go:29] interesting pod/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused" start-of-body=
Oct 30 22:15:20.336313 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:20.335801    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856" podUID="6674b9d4-017b-4109-9262-bc8ad1244712" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused"
Oct 30 22:15:20.479197 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:20.479164    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p"]
Oct 30 22:15:20.479868 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:15:20.479838    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a6d90d2_019e_41e6_9d6f_a64f537f4963.slice/crio-f8257b58ef2168158b55ceff0fe2aeed2d3fbe027f138bb1fc85eb25f6bb219e WatchSource:0}: Error finding container f8257b58ef2168158b55ceff0fe2aeed2d3fbe027f138bb1fc85eb25f6bb219e: Status 404 returned error can't find the container with id f8257b58ef2168158b55ceff0fe2aeed2d3fbe027f138bb1fc85eb25f6bb219e
Oct 30 22:15:20.481375 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:20.481358    4871 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:15:20.860644 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:20.860600    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p" event={"ID":"9a6d90d2-019e-41e6-9d6f-a64f537f4963","Type":"ContainerStarted","Data":"f8257b58ef2168158b55ceff0fe2aeed2d3fbe027f138bb1fc85eb25f6bb219e"}
Oct 30 22:15:21.321263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:21.319675    4871 patch_prober.go:29] interesting pod/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused" start-of-body=
Oct 30 22:15:21.321263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:21.319740    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856" podUID="6674b9d4-017b-4109-9262-bc8ad1244712" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused"
Oct 30 22:15:21.869525 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:21.869487    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p" event={"ID":"9a6d90d2-019e-41e6-9d6f-a64f537f4963","Type":"ContainerStarted","Data":"86315d3a75d46e072124bc72d96c465602d8f18dd00bd7e5c9a0aef6e26bffbf"}
Oct 30 22:15:21.869932 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:21.869682    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p"
Oct 30 22:15:21.963406 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:21.963366    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e1ff7ed9-fae1-4646-948e-aa1d82508cb0" path="/var/lib/kubelet/pods/e1ff7ed9-fae1-4646-948e-aa1d82508cb0/volumes"
Oct 30 22:15:21.988024 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:21.987970    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p" podStartSLOduration=2.070399644 podStartE2EDuration="2.98795665s" totalImagesPullingTime="917.557006ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:15:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:15:21.98644662 +0000 UTC m=+2264.163935237" watchObservedRunningTime="2026-10-30 22:15:21.98795665 +0000 UTC m=+2264.165445266"
Oct 30 22:15:22.361318 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:22.326380    4871 patch_prober.go:29] interesting pod/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused" start-of-body=
Oct 30 22:15:22.361318 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:22.326467    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856" podUID="6674b9d4-017b-4109-9262-bc8ad1244712" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused"
Oct 30 22:15:23.340287 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:23.325557    4871 patch_prober.go:29] interesting pod/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused" start-of-body=
Oct 30 22:15:23.340287 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:23.325634    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856" podUID="6674b9d4-017b-4109-9262-bc8ad1244712" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused"
Oct 30 22:15:24.317763 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:24.317721    4871 patch_prober.go:29] interesting pod/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused" start-of-body=
Oct 30 22:15:24.318070 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:24.318038    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856" podUID="6674b9d4-017b-4109-9262-bc8ad1244712" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused"
Oct 30 22:15:24.587750 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:24.587637    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:25.323326 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:25.323289    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"
Oct 30 22:15:25.324557 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:25.324534    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"
Oct 30 22:15:27.883834 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:27.883803    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p"
Oct 30 22:15:29.323260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:29.322768    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:34.104262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:34.104199    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:35.573484 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:15:35.573433    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:15:38.030501 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:38.030461    4871 generic.go:356] "Generic (PLEG): container finished" podID="4a9121ce-2d96-4f01-8463-ef1768b5f588" containerID="25b482f2f890c986648d162aa82289d0f3e199d77e465e88f36e4bc5830d74fd" exitCode=0
Oct 30 22:15:38.030942 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:38.030529    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4" event={"ID":"4a9121ce-2d96-4f01-8463-ef1768b5f588","Type":"ContainerDied","Data":"25b482f2f890c986648d162aa82289d0f3e199d77e465e88f36e4bc5830d74fd"}
Oct 30 22:15:38.031010 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:38.030995    4871 scope.go:122] "RemoveContainer" containerID="25b482f2f890c986648d162aa82289d0f3e199d77e465e88f36e4bc5830d74fd"
Oct 30 22:15:38.032742 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:38.032714    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:15:38.032822 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:38.032784    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:15:38.032822 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:38.032814    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:15:39.042631 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:39.042596    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4" event={"ID":"4a9121ce-2d96-4f01-8463-ef1768b5f588","Type":"ContainerStarted","Data":"600a071749ac5e0db389e71f1fc3831ab4e7ca9421ad5a984cf10b06d9f56705"}
Oct 30 22:15:39.869126 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:39.869081    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"
Oct 30 22:15:39.870397 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:39.870365    4871 patch_prober.go:29] interesting pod/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.145:15021/healthz/ready\": dial tcp 10.133.0.145:15021: connect: connection refused" start-of-body=
Oct 30 22:15:39.870550 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:39.870427    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4" podUID="4a9121ce-2d96-4f01-8463-ef1768b5f588" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.145:15021/healthz/ready\": dial tcp 10.133.0.145:15021: connect: connection refused"
Oct 30 22:15:40.330320 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:40.329424    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:40.497224 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:40.497192    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"
Oct 30 22:15:40.869270 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:40.869206    4871 patch_prober.go:29] interesting pod/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.145:15021/healthz/ready\": dial tcp 10.133.0.145:15021: connect: connection refused" start-of-body=
Oct 30 22:15:40.869466 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:40.869296    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4" podUID="4a9121ce-2d96-4f01-8463-ef1768b5f588" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.145:15021/healthz/ready\": dial tcp 10.133.0.145:15021: connect: connection refused"
Oct 30 22:15:41.597739 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:41.597706    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:41.869275 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:41.869204    4871 patch_prober.go:29] interesting pod/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.145:15021/healthz/ready\": dial tcp 10.133.0.145:15021: connect: connection refused" start-of-body=
Oct 30 22:15:41.869456 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:41.869297    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4" podUID="4a9121ce-2d96-4f01-8463-ef1768b5f588" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.145:15021/healthz/ready\": dial tcp 10.133.0.145:15021: connect: connection refused"
Oct 30 22:15:42.869136 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:42.869095    4871 patch_prober.go:29] interesting pod/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.145:15021/healthz/ready\": dial tcp 10.133.0.145:15021: connect: connection refused" start-of-body=
Oct 30 22:15:42.869591 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:42.869166    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4" podUID="4a9121ce-2d96-4f01-8463-ef1768b5f588" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.145:15021/healthz/ready\": dial tcp 10.133.0.145:15021: connect: connection refused"
Oct 30 22:15:43.869511 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:43.869467    4871 patch_prober.go:29] interesting pod/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.145:15021/healthz/ready\": dial tcp 10.133.0.145:15021: connect: connection refused" start-of-body=
Oct 30 22:15:43.869911 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:43.869562    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4" podUID="4a9121ce-2d96-4f01-8463-ef1768b5f588" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.145:15021/healthz/ready\": dial tcp 10.133.0.145:15021: connect: connection refused"
Oct 30 22:15:44.932716 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:44.932319    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"
Oct 30 22:15:44.934543 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:44.934517    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"
Oct 30 22:15:45.249817 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:45.249729    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:49.473998 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:49.473738    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:49.664263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:49.662371    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:49.827319 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:49.822992    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:51.777306 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:51.777267    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:56.516099 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:15:56.516049    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:01.567428 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:01.567393    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:05.568767 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:16:05.568725    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:16:06.615363 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:06.615330    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:07.591590 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:07.591554    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:07.745422 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:07.743810    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:12.907699 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:12.907660    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:17.496923 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:17.496879    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:17.721803 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:17.721761    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:20.105109 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:20.105059    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:22.114334 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:22.114295    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:22.304454 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:22.304407    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:22.529486 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:22.529388    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:22.627864 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:22.627822    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"]
Oct 30 22:16:22.628087 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:22.628062    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg" podUID="fa8f3772-b525-4512-a629-603102f076c0" containerName="istio-proxy" containerID="cri-o://7f7ead720820a6483861370e167447f3ece5c23128c2ad80b4702c27cf8149f7" gracePeriod=30
Oct 30 22:16:22.676250 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:22.676202    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"]
Oct 30 22:16:22.676740 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:22.676704    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2" podUID="9f13966e-b7ac-4a97-89b0-f98a9b44c5ec" containerName="istio-proxy" containerID="cri-o://4dc7b716daba4e00a53f2febc9bafc3220bdd3af72edc5c0acabfcc351d3cabb" gracePeriod=30
Oct 30 22:16:22.682491 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:22.682459    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"]
Oct 30 22:16:22.682761 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:22.682710    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj" podUID="fd6f1baa-ab90-488d-abe5-e70b4f6298ce" containerName="istio-proxy" containerID="cri-o://3d0c97ba29ffb11d992dbcdaf977a73a698ed2d410a78d19bc6cc036b07bdbd5" gracePeriod=30
Oct 30 22:16:23.619332 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:23.619288    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:24.543312 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:24.543274    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:27.313603 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:27.313570    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:27.965253 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:27.965207    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"
Oct 30 22:16:28.002370 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.002337    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"
Oct 30 22:16:28.021512 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.021468    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-istiod-ca-cert\") pod \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\" (UID: \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\") "
Oct 30 22:16:28.021713 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.021549    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-istio-envoy\") pod \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\" (UID: \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\") "
Oct 30 22:16:28.021713 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.021618    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-workload-socket\") pod \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\" (UID: \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\") "
Oct 30 22:16:28.021713 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.021694    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-istio-data\") pod \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\" (UID: \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\") "
Oct 30 22:16:28.021871 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.021748    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-istio-token\" (UniqueName: \"kubernetes.io/projected/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-istio-token\") pod \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\" (UID: \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\") "
Oct 30 22:16:28.021871 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.021821    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-credential-socket\") pod \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\" (UID: \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\") "
Oct 30 22:16:28.021871 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.021862    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-kube-api-access-6zckr\" (UniqueName: \"kubernetes.io/projected/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-kube-api-access-6zckr\") pod \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\" (UID: \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\") "
Oct 30 22:16:28.022028 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.021855    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-istiod-ca-cert" pod "9f13966e-b7ac-4a97-89b0-f98a9b44c5ec" (UID: "9f13966e-b7ac-4a97-89b0-f98a9b44c5ec"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:16:28.022028 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.021985    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-workload-certs\") pod \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\" (UID: \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\") "
Oct 30 22:16:28.022028 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.022017    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-istio-podinfo\") pod \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\" (UID: \"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec\") "
Oct 30 22:16:28.022190 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.022143    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-istio-data" pod "9f13966e-b7ac-4a97-89b0-f98a9b44c5ec" (UID: "9f13966e-b7ac-4a97-89b0-f98a9b44c5ec"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:28.022190 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.022162    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-credential-socket" pod "9f13966e-b7ac-4a97-89b0-f98a9b44c5ec" (UID: "9f13966e-b7ac-4a97-89b0-f98a9b44c5ec"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:28.022374 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.022359    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-workload-certs" pod "9f13966e-b7ac-4a97-89b0-f98a9b44c5ec" (UID: "9f13966e-b7ac-4a97-89b0-f98a9b44c5ec"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:28.022793 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.022758    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:16:28.022793 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.022789    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:16:28.023010 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.022805    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:16:28.023010 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.022819    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:16:28.023372 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.023342    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-workload-socket" pod "9f13966e-b7ac-4a97-89b0-f98a9b44c5ec" (UID: "9f13966e-b7ac-4a97-89b0-f98a9b44c5ec"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:28.024493 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.024471    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-istio-podinfo" pod "9f13966e-b7ac-4a97-89b0-f98a9b44c5ec" (UID: "9f13966e-b7ac-4a97-89b0-f98a9b44c5ec"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:16:28.024631 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.024606    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-kube-api-access-6zckr" pod "9f13966e-b7ac-4a97-89b0-f98a9b44c5ec" (UID: "9f13966e-b7ac-4a97-89b0-f98a9b44c5ec"). InnerVolumeSpecName "kube-api-access-6zckr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:28.025206 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.025185    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-istio-token" pod "9f13966e-b7ac-4a97-89b0-f98a9b44c5ec" (UID: "9f13966e-b7ac-4a97-89b0-f98a9b44c5ec"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:28.025638 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.025621    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-istio-envoy" pod "9f13966e-b7ac-4a97-89b0-f98a9b44c5ec" (UID: "9f13966e-b7ac-4a97-89b0-f98a9b44c5ec"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:28.034187 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.034165    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"
Oct 30 22:16:28.123279 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.123230    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fa8f3772-b525-4512-a629-603102f076c0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa8f3772-b525-4512-a629-603102f076c0-credential-socket\") pod \"fa8f3772-b525-4512-a629-603102f076c0\" (UID: \"fa8f3772-b525-4512-a629-603102f076c0\") "
Oct 30 22:16:28.123478 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.123303    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-istio-token\" (UniqueName: \"kubernetes.io/projected/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-istio-token\") pod \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\" (UID: \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\") "
Oct 30 22:16:28.123478 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.123375    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-istiod-ca-cert\") pod \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\" (UID: \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\") "
Oct 30 22:16:28.123478 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.123395    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-istio-podinfo\") pod \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\" (UID: \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\") "
Oct 30 22:16:28.123478 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.123438    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fa8f3772-b525-4512-a629-603102f076c0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa8f3772-b525-4512-a629-603102f076c0-workload-certs\") pod \"fa8f3772-b525-4512-a629-603102f076c0\" (UID: \"fa8f3772-b525-4512-a629-603102f076c0\") "
Oct 30 22:16:28.123478 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.123457    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fa8f3772-b525-4512-a629-603102f076c0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa8f3772-b525-4512-a629-603102f076c0-istiod-ca-cert\") pod \"fa8f3772-b525-4512-a629-603102f076c0\" (UID: \"fa8f3772-b525-4512-a629-603102f076c0\") "
Oct 30 22:16:28.123768 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.123507    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fa8f3772-b525-4512-a629-603102f076c0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa8f3772-b525-4512-a629-603102f076c0-workload-socket\") pod \"fa8f3772-b525-4512-a629-603102f076c0\" (UID: \"fa8f3772-b525-4512-a629-603102f076c0\") "
Oct 30 22:16:28.123768 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.123522    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa8f3772-b525-4512-a629-603102f076c0-credential-socket" pod "fa8f3772-b525-4512-a629-603102f076c0" (UID: "fa8f3772-b525-4512-a629-603102f076c0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:28.123768 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.123552    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fa8f3772-b525-4512-a629-603102f076c0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa8f3772-b525-4512-a629-603102f076c0-istio-envoy\") pod \"fa8f3772-b525-4512-a629-603102f076c0\" (UID: \"fa8f3772-b525-4512-a629-603102f076c0\") "
Oct 30 22:16:28.123768 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.123593    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fa8f3772-b525-4512-a629-603102f076c0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa8f3772-b525-4512-a629-603102f076c0-istio-data\") pod \"fa8f3772-b525-4512-a629-603102f076c0\" (UID: \"fa8f3772-b525-4512-a629-603102f076c0\") "
Oct 30 22:16:28.123768 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.123619    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fa8f3772-b525-4512-a629-603102f076c0-istio-token\" (UniqueName: \"kubernetes.io/projected/fa8f3772-b525-4512-a629-603102f076c0-istio-token\") pod \"fa8f3772-b525-4512-a629-603102f076c0\" (UID: \"fa8f3772-b525-4512-a629-603102f076c0\") "
Oct 30 22:16:28.123768 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.123675    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-workload-socket\") pod \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\" (UID: \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\") "
Oct 30 22:16:28.123768 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.123707    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-istio-data\") pod \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\" (UID: \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\") "
Oct 30 22:16:28.124134 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.123776    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-credential-socket\") pod \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\" (UID: \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\") "
Oct 30 22:16:28.124134 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.123853    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-workload-certs\") pod \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\" (UID: \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\") "
Oct 30 22:16:28.124134 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.123856    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-istiod-ca-cert" pod "fd6f1baa-ab90-488d-abe5-e70b4f6298ce" (UID: "fd6f1baa-ab90-488d-abe5-e70b4f6298ce"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:16:28.124134 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.123900    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-kube-api-access-6wdh5\" (UniqueName: \"kubernetes.io/projected/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-kube-api-access-6wdh5\") pod \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\" (UID: \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\") "
Oct 30 22:16:28.124134 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.123907    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa8f3772-b525-4512-a629-603102f076c0-istiod-ca-cert" pod "fa8f3772-b525-4512-a629-603102f076c0" (UID: "fa8f3772-b525-4512-a629-603102f076c0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:16:28.124134 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.123934    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fa8f3772-b525-4512-a629-603102f076c0-kube-api-access-fjpfg\" (UniqueName: \"kubernetes.io/projected/fa8f3772-b525-4512-a629-603102f076c0-kube-api-access-fjpfg\") pod \"fa8f3772-b525-4512-a629-603102f076c0\" (UID: \"fa8f3772-b525-4512-a629-603102f076c0\") "
Oct 30 22:16:28.124134 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.123963    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fa8f3772-b525-4512-a629-603102f076c0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa8f3772-b525-4512-a629-603102f076c0-istio-podinfo\") pod \"fa8f3772-b525-4512-a629-603102f076c0\" (UID: \"fa8f3772-b525-4512-a629-603102f076c0\") "
Oct 30 22:16:28.124134 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.124016    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa8f3772-b525-4512-a629-603102f076c0-istio-data" pod "fa8f3772-b525-4512-a629-603102f076c0" (UID: "fa8f3772-b525-4512-a629-603102f076c0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:28.124134 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.124042    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-istio-envoy\") pod \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\" (UID: \"fd6f1baa-ab90-488d-abe5-e70b4f6298ce\") "
Oct 30 22:16:28.124134 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.124065    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa8f3772-b525-4512-a629-603102f076c0-workload-certs" pod "fa8f3772-b525-4512-a629-603102f076c0" (UID: "fa8f3772-b525-4512-a629-603102f076c0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:28.124134 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.124135    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa8f3772-b525-4512-a629-603102f076c0-workload-socket" pod "fa8f3772-b525-4512-a629-603102f076c0" (UID: "fa8f3772-b525-4512-a629-603102f076c0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:28.124714 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.124219    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-credential-socket" pod "fd6f1baa-ab90-488d-abe5-e70b4f6298ce" (UID: "fd6f1baa-ab90-488d-abe5-e70b4f6298ce"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:28.124714 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.124447    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-workload-socket" pod "fd6f1baa-ab90-488d-abe5-e70b4f6298ce" (UID: "fd6f1baa-ab90-488d-abe5-e70b4f6298ce"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:28.124820 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.124756    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:16:28.124820 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.124776    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:16:28.124820 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.124793    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa8f3772-b525-4512-a629-603102f076c0-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:16:28.124977 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.124841    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa8f3772-b525-4512-a629-603102f076c0-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:16:28.124977 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.124860    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6zckr\" (UniqueName: \"kubernetes.io/projected/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-kube-api-access-6zckr\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:16:28.124977 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.124876    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa8f3772-b525-4512-a629-603102f076c0-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:16:28.124977 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.124890    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa8f3772-b525-4512-a629-603102f076c0-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:16:28.124977 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.124890    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-workload-certs" pod "fd6f1baa-ab90-488d-abe5-e70b4f6298ce" (UID: "fd6f1baa-ab90-488d-abe5-e70b4f6298ce"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:28.124977 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.124928    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:16:28.124977 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.124934    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-istio-data" pod "fd6f1baa-ab90-488d-abe5-e70b4f6298ce" (UID: "fd6f1baa-ab90-488d-abe5-e70b4f6298ce"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:28.124977 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.124944    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:16:28.124977 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.124961    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:16:28.124977 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.124976    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:16:28.125544 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.124990    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa8f3772-b525-4512-a629-603102f076c0-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:16:28.125544 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.125004    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:16:28.127110 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.126906    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-istio-envoy" pod "fd6f1baa-ab90-488d-abe5-e70b4f6298ce" (UID: "fd6f1baa-ab90-488d-abe5-e70b4f6298ce"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:28.127110 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.126955    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa8f3772-b525-4512-a629-603102f076c0-istio-token" pod "fa8f3772-b525-4512-a629-603102f076c0" (UID: "fa8f3772-b525-4512-a629-603102f076c0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:28.127110 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.126987    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa8f3772-b525-4512-a629-603102f076c0-kube-api-access-fjpfg" pod "fa8f3772-b525-4512-a629-603102f076c0" (UID: "fa8f3772-b525-4512-a629-603102f076c0"). InnerVolumeSpecName "kube-api-access-fjpfg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:28.127110 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.127054    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa8f3772-b525-4512-a629-603102f076c0-istio-envoy" pod "fa8f3772-b525-4512-a629-603102f076c0" (UID: "fa8f3772-b525-4512-a629-603102f076c0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:28.127110 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.127078    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-istio-token" pod "fd6f1baa-ab90-488d-abe5-e70b4f6298ce" (UID: "fd6f1baa-ab90-488d-abe5-e70b4f6298ce"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:28.127449 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.127147    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-kube-api-access-6wdh5" pod "fd6f1baa-ab90-488d-abe5-e70b4f6298ce" (UID: "fd6f1baa-ab90-488d-abe5-e70b4f6298ce"). InnerVolumeSpecName "kube-api-access-6wdh5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:28.128140 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.128113    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fa8f3772-b525-4512-a629-603102f076c0-istio-podinfo" pod "fa8f3772-b525-4512-a629-603102f076c0" (UID: "fa8f3772-b525-4512-a629-603102f076c0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:16:28.129365 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.129336    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-istio-podinfo" pod "fd6f1baa-ab90-488d-abe5-e70b4f6298ce" (UID: "fd6f1baa-ab90-488d-abe5-e70b4f6298ce"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:16:28.226731 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.226622    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:16:28.226731 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.226661    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6wdh5\" (UniqueName: \"kubernetes.io/projected/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-kube-api-access-6wdh5\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:16:28.226731 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.226676    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fjpfg\" (UniqueName: \"kubernetes.io/projected/fa8f3772-b525-4512-a629-603102f076c0-kube-api-access-fjpfg\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:16:28.226731 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.226689    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa8f3772-b525-4512-a629-603102f076c0-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:16:28.226731 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.226701    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:16:28.226731 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.226713    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:16:28.226731 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.226727    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:16:28.227124 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.226739    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa8f3772-b525-4512-a629-603102f076c0-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:16:28.227124 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.226751    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa8f3772-b525-4512-a629-603102f076c0-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:16:28.227124 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.226766    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd6f1baa-ab90-488d-abe5-e70b4f6298ce-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:16:28.556196 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.556097    4871 generic.go:356] "Generic (PLEG): container finished" podID="fa8f3772-b525-4512-a629-603102f076c0" containerID="7f7ead720820a6483861370e167447f3ece5c23128c2ad80b4702c27cf8149f7" exitCode=0
Oct 30 22:16:28.556196 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.556147    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"
Oct 30 22:16:28.556196 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.556171    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg" event={"ID":"fa8f3772-b525-4512-a629-603102f076c0","Type":"ContainerDied","Data":"7f7ead720820a6483861370e167447f3ece5c23128c2ad80b4702c27cf8149f7"}
Oct 30 22:16:28.556801 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.556231    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg" event={"ID":"fa8f3772-b525-4512-a629-603102f076c0","Type":"ContainerDied","Data":"ddad1ddd23d08b26ebfd6fbca87f14bab367732c94cdcfc448ff95378a7d7ae6"}
Oct 30 22:16:28.556801 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.556294    4871 scope.go:122] "RemoveContainer" containerID="7f7ead720820a6483861370e167447f3ece5c23128c2ad80b4702c27cf8149f7"
Oct 30 22:16:28.563481 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.563454    4871 generic.go:356] "Generic (PLEG): container finished" podID="9f13966e-b7ac-4a97-89b0-f98a9b44c5ec" containerID="4dc7b716daba4e00a53f2febc9bafc3220bdd3af72edc5c0acabfcc351d3cabb" exitCode=0
Oct 30 22:16:28.563636 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.563609    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"
Oct 30 22:16:28.564164 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.564135    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2" event={"ID":"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec","Type":"ContainerDied","Data":"4dc7b716daba4e00a53f2febc9bafc3220bdd3af72edc5c0acabfcc351d3cabb"}
Oct 30 22:16:28.564339 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.564197    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2" event={"ID":"9f13966e-b7ac-4a97-89b0-f98a9b44c5ec","Type":"ContainerDied","Data":"9e5e93eb3b38d5e421bed4cbd85656bd95d8bbbe7ccc4e19e600996e9bfca7a3"}
Oct 30 22:16:28.567337 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.567305    4871 generic.go:356] "Generic (PLEG): container finished" podID="fd6f1baa-ab90-488d-abe5-e70b4f6298ce" containerID="3d0c97ba29ffb11d992dbcdaf977a73a698ed2d410a78d19bc6cc036b07bdbd5" exitCode=0
Oct 30 22:16:28.567477 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.567351    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"
Oct 30 22:16:28.567477 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.567401    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj" event={"ID":"fd6f1baa-ab90-488d-abe5-e70b4f6298ce","Type":"ContainerDied","Data":"3d0c97ba29ffb11d992dbcdaf977a73a698ed2d410a78d19bc6cc036b07bdbd5"}
Oct 30 22:16:28.567477 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.567463    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj" event={"ID":"fd6f1baa-ab90-488d-abe5-e70b4f6298ce","Type":"ContainerDied","Data":"014bfbc48312a70ef299b38bf05ade33d17794c6bd4217276d9b88bd1da2ea0f"}
Oct 30 22:16:28.569820 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.569793    4871 scope.go:122] "RemoveContainer" containerID="7f7ead720820a6483861370e167447f3ece5c23128c2ad80b4702c27cf8149f7"
Oct 30 22:16:28.570150 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:16:28.570121    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f7ead720820a6483861370e167447f3ece5c23128c2ad80b4702c27cf8149f7\": container with ID starting with 7f7ead720820a6483861370e167447f3ece5c23128c2ad80b4702c27cf8149f7 not found: ID does not exist" containerID="7f7ead720820a6483861370e167447f3ece5c23128c2ad80b4702c27cf8149f7"
Oct 30 22:16:28.570150 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.570160    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f7ead720820a6483861370e167447f3ece5c23128c2ad80b4702c27cf8149f7"} err="failed to get container status \"7f7ead720820a6483861370e167447f3ece5c23128c2ad80b4702c27cf8149f7\": rpc error: code = NotFound desc = could not find container \"7f7ead720820a6483861370e167447f3ece5c23128c2ad80b4702c27cf8149f7\": container with ID starting with 7f7ead720820a6483861370e167447f3ece5c23128c2ad80b4702c27cf8149f7 not found: ID does not exist"
Oct 30 22:16:28.570339 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.570188    4871 scope.go:122] "RemoveContainer" containerID="4dc7b716daba4e00a53f2febc9bafc3220bdd3af72edc5c0acabfcc351d3cabb"
Oct 30 22:16:28.593408 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.593374    4871 scope.go:122] "RemoveContainer" containerID="4dc7b716daba4e00a53f2febc9bafc3220bdd3af72edc5c0acabfcc351d3cabb"
Oct 30 22:16:28.593775 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:16:28.593751    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4dc7b716daba4e00a53f2febc9bafc3220bdd3af72edc5c0acabfcc351d3cabb\": container with ID starting with 4dc7b716daba4e00a53f2febc9bafc3220bdd3af72edc5c0acabfcc351d3cabb not found: ID does not exist" containerID="4dc7b716daba4e00a53f2febc9bafc3220bdd3af72edc5c0acabfcc351d3cabb"
Oct 30 22:16:28.593839 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.593785    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4dc7b716daba4e00a53f2febc9bafc3220bdd3af72edc5c0acabfcc351d3cabb"} err="failed to get container status \"4dc7b716daba4e00a53f2febc9bafc3220bdd3af72edc5c0acabfcc351d3cabb\": rpc error: code = NotFound desc = could not find container \"4dc7b716daba4e00a53f2febc9bafc3220bdd3af72edc5c0acabfcc351d3cabb\": container with ID starting with 4dc7b716daba4e00a53f2febc9bafc3220bdd3af72edc5c0acabfcc351d3cabb not found: ID does not exist"
Oct 30 22:16:28.593839 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.593804    4871 scope.go:122] "RemoveContainer" containerID="3d0c97ba29ffb11d992dbcdaf977a73a698ed2d410a78d19bc6cc036b07bdbd5"
Oct 30 22:16:28.604932 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.604902    4871 scope.go:122] "RemoveContainer" containerID="3d0c97ba29ffb11d992dbcdaf977a73a698ed2d410a78d19bc6cc036b07bdbd5"
Oct 30 22:16:28.605352 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:16:28.605330    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d0c97ba29ffb11d992dbcdaf977a73a698ed2d410a78d19bc6cc036b07bdbd5\": container with ID starting with 3d0c97ba29ffb11d992dbcdaf977a73a698ed2d410a78d19bc6cc036b07bdbd5 not found: ID does not exist" containerID="3d0c97ba29ffb11d992dbcdaf977a73a698ed2d410a78d19bc6cc036b07bdbd5"
Oct 30 22:16:28.605429 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.605363    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d0c97ba29ffb11d992dbcdaf977a73a698ed2d410a78d19bc6cc036b07bdbd5"} err="failed to get container status \"3d0c97ba29ffb11d992dbcdaf977a73a698ed2d410a78d19bc6cc036b07bdbd5\": rpc error: code = NotFound desc = could not find container \"3d0c97ba29ffb11d992dbcdaf977a73a698ed2d410a78d19bc6cc036b07bdbd5\": container with ID starting with 3d0c97ba29ffb11d992dbcdaf977a73a698ed2d410a78d19bc6cc036b07bdbd5 not found: ID does not exist"
Oct 30 22:16:28.720559 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.720520    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"]
Oct 30 22:16:28.766188 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.766148    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-jbljg"]
Oct 30 22:16:28.828816 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.828731    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"]
Oct 30 22:16:28.875350 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.875312    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-85wfj"]
Oct 30 22:16:28.939930 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.939889    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"]
Oct 30 22:16:28.983876 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:28.983836    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cvb2"]
Oct 30 22:16:29.961322 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:29.961290    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9f13966e-b7ac-4a97-89b0-f98a9b44c5ec" path="/var/lib/kubelet/pods/9f13966e-b7ac-4a97-89b0-f98a9b44c5ec/volumes"
Oct 30 22:16:29.961879 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:29.961859    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fa8f3772-b525-4512-a629-603102f076c0" path="/var/lib/kubelet/pods/fa8f3772-b525-4512-a629-603102f076c0/volumes"
Oct 30 22:16:29.962371 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:29.962351    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fd6f1baa-ab90-488d-abe5-e70b4f6298ce" path="/var/lib/kubelet/pods/fd6f1baa-ab90-488d-abe5-e70b4f6298ce/volumes"
Oct 30 22:16:32.167068 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:32.167028    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:33.501884 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:33.501841    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:35.570224 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:16:35.570182    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:16:38.309139 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:38.309102    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:41.943005 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:41.942970    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:43.082438 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:43.082395    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:46.025387 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:46.025347    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:46.233055 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:46.233008    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:46.390472 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:46.390439    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:47.934125 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:47.934085    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:54.218319 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:54.218285    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:57.578522 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:57.578485    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:58.067074 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:58.067040    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:59.948960 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:16:59.948900    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:01.439628 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:01.439584    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:01.700328 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:01.700208    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:01.951419 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:01.951319    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:02.529409 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:02.529373    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:05.570690 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:17:05.570650    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:17:08.471078 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:08.471043    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:09.060660 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:09.060618    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"]
Oct 30 22:17:09.060914 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:09.060861    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7" podUID="96df7172-8d94-4726-997e-2b0c9aeb8f5d" containerName="istio-proxy" containerID="cri-o://18a1b8d2852d239fe6b9af99f128bd5c89bee4de0808f52bac955f5f9bea7a63" gracePeriod=30
Oct 30 22:17:09.756997 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:09.756956    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:10.104112 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.104010    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:10.574335 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.574286    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"]
Oct 30 22:17:10.574993 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.574976    4871 cpu_manager.go:414] "containerMap: removing container" podUID="29cba590-3fb8-4d7c-a357-aebb92f595cb" containerName="cleanup"
Oct 30 22:17:10.575044 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.574994    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="29cba590-3fb8-4d7c-a357-aebb92f595cb" containerName="cleanup"
Oct 30 22:17:10.575044 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.575035    4871 cpu_manager.go:414] "containerMap: removing container" podUID="9f13966e-b7ac-4a97-89b0-f98a9b44c5ec" containerName="istio-proxy"
Oct 30 22:17:10.575044 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.575041    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9f13966e-b7ac-4a97-89b0-f98a9b44c5ec" containerName="istio-proxy"
Oct 30 22:17:10.575150 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.575063    4871 cpu_manager.go:414] "containerMap: removing container" podUID="fd6f1baa-ab90-488d-abe5-e70b4f6298ce" containerName="istio-proxy"
Oct 30 22:17:10.575150 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.575069    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fd6f1baa-ab90-488d-abe5-e70b4f6298ce" containerName="istio-proxy"
Oct 30 22:17:10.575150 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.575077    4871 cpu_manager.go:414] "containerMap: removing container" podUID="fa8f3772-b525-4512-a629-603102f076c0" containerName="istio-proxy"
Oct 30 22:17:10.575150 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.575082    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fa8f3772-b525-4512-a629-603102f076c0" containerName="istio-proxy"
Oct 30 22:17:10.575287 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.575165    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="9f13966e-b7ac-4a97-89b0-f98a9b44c5ec" containerName="istio-proxy"
Oct 30 22:17:10.575287 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.575172    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="e1ff7ed9-fae1-4646-948e-aa1d82508cb0" containerName="cleanup"
Oct 30 22:17:10.575287 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.575181    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="fd6f1baa-ab90-488d-abe5-e70b4f6298ce" containerName="istio-proxy"
Oct 30 22:17:10.575287 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.575187    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="fa8f3772-b525-4512-a629-603102f076c0" containerName="istio-proxy"
Oct 30 22:17:10.579045 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.579022    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"
Oct 30 22:17:10.632409 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.632366    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-63a282-openshift-default-dockercfg-8bk4v\""
Oct 30 22:17:10.711331 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.687919    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgb48\" (UniqueName: \"kubernetes.io/projected/cf4abc58-fe24-486b-a195-1eb2a3b2e643-kube-api-access-rgb48\") pod \"e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2\" (UID: \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\") " pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"
Oct 30 22:17:10.711685 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.711650    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf4abc58-fe24-486b-a195-1eb2a3b2e643-istio-podinfo\") pod \"e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2\" (UID: \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\") " pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"
Oct 30 22:17:10.711839 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.711818    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf4abc58-fe24-486b-a195-1eb2a3b2e643-istio-token\") pod \"e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2\" (UID: \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\") " pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"
Oct 30 22:17:10.711975 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.711958    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf4abc58-fe24-486b-a195-1eb2a3b2e643-istiod-ca-cert\") pod \"e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2\" (UID: \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\") " pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"
Oct 30 22:17:10.712096 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.712081    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf4abc58-fe24-486b-a195-1eb2a3b2e643-istio-data\") pod \"e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2\" (UID: \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\") " pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"
Oct 30 22:17:10.712350 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.712331    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf4abc58-fe24-486b-a195-1eb2a3b2e643-istio-envoy\") pod \"e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2\" (UID: \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\") " pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"
Oct 30 22:17:10.712509 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.712492    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf4abc58-fe24-486b-a195-1eb2a3b2e643-workload-certs\") pod \"e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2\" (UID: \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\") " pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"
Oct 30 22:17:10.712775 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.712756    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf4abc58-fe24-486b-a195-1eb2a3b2e643-workload-socket\") pod \"e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2\" (UID: \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\") " pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"
Oct 30 22:17:10.712917 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.712902    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf4abc58-fe24-486b-a195-1eb2a3b2e643-credential-socket\") pod \"e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2\" (UID: \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\") " pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"
Oct 30 22:17:10.759937 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.759893    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:10.853312 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.852869    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"]
Oct 30 22:17:10.873271 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.869214    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf4abc58-fe24-486b-a195-1eb2a3b2e643-workload-socket\") pod \"e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2\" (UID: \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\") " pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"
Oct 30 22:17:10.873271 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.869298    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf4abc58-fe24-486b-a195-1eb2a3b2e643-credential-socket\") pod \"e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2\" (UID: \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\") " pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"
Oct 30 22:17:10.873271 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.869346    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgb48\" (UniqueName: \"kubernetes.io/projected/cf4abc58-fe24-486b-a195-1eb2a3b2e643-kube-api-access-rgb48\") pod \"e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2\" (UID: \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\") " pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"
Oct 30 22:17:10.873271 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.869381    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf4abc58-fe24-486b-a195-1eb2a3b2e643-istio-podinfo\") pod \"e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2\" (UID: \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\") " pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"
Oct 30 22:17:10.873271 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.869409    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf4abc58-fe24-486b-a195-1eb2a3b2e643-istio-token\") pod \"e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2\" (UID: \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\") " pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"
Oct 30 22:17:10.873271 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.869442    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf4abc58-fe24-486b-a195-1eb2a3b2e643-istiod-ca-cert\") pod \"e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2\" (UID: \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\") " pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"
Oct 30 22:17:10.873271 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.869484    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf4abc58-fe24-486b-a195-1eb2a3b2e643-istio-data\") pod \"e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2\" (UID: \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\") " pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"
Oct 30 22:17:10.873271 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.869615    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf4abc58-fe24-486b-a195-1eb2a3b2e643-istio-envoy\") pod \"e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2\" (UID: \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\") " pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"
Oct 30 22:17:10.873271 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.869663    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf4abc58-fe24-486b-a195-1eb2a3b2e643-workload-certs\") pod \"e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2\" (UID: \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\") " pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"
Oct 30 22:17:10.873271 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.870185    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf4abc58-fe24-486b-a195-1eb2a3b2e643-workload-certs\") pod \"e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2\" (UID: \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\") " pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"
Oct 30 22:17:10.873271 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.870546    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf4abc58-fe24-486b-a195-1eb2a3b2e643-workload-socket\") pod \"e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2\" (UID: \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\") " pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"
Oct 30 22:17:10.873271 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.870799    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf4abc58-fe24-486b-a195-1eb2a3b2e643-credential-socket\") pod \"e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2\" (UID: \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\") " pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"
Oct 30 22:17:10.880362 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.876895    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf4abc58-fe24-486b-a195-1eb2a3b2e643-istiod-ca-cert\") pod \"e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2\" (UID: \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\") " pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"
Oct 30 22:17:10.880362 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.877527    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf4abc58-fe24-486b-a195-1eb2a3b2e643-istio-data\") pod \"e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2\" (UID: \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\") " pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"
Oct 30 22:17:10.912827 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.903930    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf4abc58-fe24-486b-a195-1eb2a3b2e643-istio-podinfo\") pod \"e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2\" (UID: \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\") " pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"
Oct 30 22:17:10.931249 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.930736    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf4abc58-fe24-486b-a195-1eb2a3b2e643-istio-envoy\") pod \"e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2\" (UID: \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\") " pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"
Oct 30 22:17:10.932328 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.932290    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:10.933887 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.933865    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgb48\" (UniqueName: \"kubernetes.io/projected/cf4abc58-fe24-486b-a195-1eb2a3b2e643-kube-api-access-rgb48\") pod \"e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2\" (UID: \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\") " pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"
Oct 30 22:17:10.934824 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:10.934804    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf4abc58-fe24-486b-a195-1eb2a3b2e643-istio-token\") pod \"e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2\" (UID: \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\") " pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"
Oct 30 22:17:11.193565 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:11.193525    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"
Oct 30 22:17:11.624744 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:11.624707    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"]
Oct 30 22:17:11.625328 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:17:11.625301    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf4abc58_fe24_486b_a195_1eb2a3b2e643.slice/crio-dd32447f7f0073cfea07ebf84049a6a102f06f307320c738f8549dfb34e14729 WatchSource:0}: Error finding container dd32447f7f0073cfea07ebf84049a6a102f06f307320c738f8549dfb34e14729: Status 404 returned error can't find the container with id dd32447f7f0073cfea07ebf84049a6a102f06f307320c738f8549dfb34e14729
Oct 30 22:17:11.633629 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:11.633583    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:17:11.633751 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:11.633675    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:17:11.633751 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:11.633717    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:17:11.878320 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:11.878227    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:11.995187 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:11.995148    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2" event={"ID":"cf4abc58-fe24-486b-a195-1eb2a3b2e643","Type":"ContainerStarted","Data":"0cbc0d71509db4413284c81b3a059c69a31415cd63a8b165fe83724ac10162f9"}
Oct 30 22:17:11.995380 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:11.995210    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2" event={"ID":"cf4abc58-fe24-486b-a195-1eb2a3b2e643","Type":"ContainerStarted","Data":"dd32447f7f0073cfea07ebf84049a6a102f06f307320c738f8549dfb34e14729"}
Oct 30 22:17:12.127290 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:12.122864    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2" podStartSLOduration=2.122801733 podStartE2EDuration="2.122801733s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:17:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:17:12.118960374 +0000 UTC m=+2374.296448992" watchObservedRunningTime="2026-10-30 22:17:12.122801733 +0000 UTC m=+2374.300290350"
Oct 30 22:17:12.194269 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:12.194128    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"
Oct 30 22:17:12.195687 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:12.195657    4871 patch_prober.go:29] interesting pod/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused" start-of-body=
Oct 30 22:17:12.195829 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:12.195717    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2" podUID="cf4abc58-fe24-486b-a195-1eb2a3b2e643" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused"
Oct 30 22:17:12.366874 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:12.366821    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:13.199328 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:13.199296    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"
Oct 30 22:17:14.013301 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:14.013256    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"
Oct 30 22:17:14.014428 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:14.014403    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"
Oct 30 22:17:14.392438 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:14.392409    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"
Oct 30 22:17:14.429849 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:14.429811    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96df7172-8d94-4726-997e-2b0c9aeb8f5d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96df7172-8d94-4726-997e-2b0c9aeb8f5d-workload-socket\") pod \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\" (UID: \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\") "
Oct 30 22:17:14.430017 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:14.429864    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/96df7172-8d94-4726-997e-2b0c9aeb8f5d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96df7172-8d94-4726-997e-2b0c9aeb8f5d-istiod-ca-cert\") pod \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\" (UID: \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\") "
Oct 30 22:17:14.430017 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:14.429890    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96df7172-8d94-4726-997e-2b0c9aeb8f5d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96df7172-8d94-4726-997e-2b0c9aeb8f5d-credential-socket\") pod \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\" (UID: \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\") "
Oct 30 22:17:14.430017 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:14.429915    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96df7172-8d94-4726-997e-2b0c9aeb8f5d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96df7172-8d94-4726-997e-2b0c9aeb8f5d-workload-certs\") pod \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\" (UID: \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\") "
Oct 30 22:17:14.430017 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:14.429982    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96df7172-8d94-4726-997e-2b0c9aeb8f5d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96df7172-8d94-4726-997e-2b0c9aeb8f5d-istio-envoy\") pod \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\" (UID: \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\") "
Oct 30 22:17:14.430017 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:14.430016    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/96df7172-8d94-4726-997e-2b0c9aeb8f5d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96df7172-8d94-4726-997e-2b0c9aeb8f5d-istio-podinfo\") pod \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\" (UID: \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\") "
Oct 30 22:17:14.430534 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:14.430109    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96df7172-8d94-4726-997e-2b0c9aeb8f5d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96df7172-8d94-4726-997e-2b0c9aeb8f5d-istio-data\") pod \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\" (UID: \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\") "
Oct 30 22:17:14.430534 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:14.430170    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/96df7172-8d94-4726-997e-2b0c9aeb8f5d-istio-token\" (UniqueName: \"kubernetes.io/projected/96df7172-8d94-4726-997e-2b0c9aeb8f5d-istio-token\") pod \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\" (UID: \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\") "
Oct 30 22:17:14.430534 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:14.430196    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96df7172-8d94-4726-997e-2b0c9aeb8f5d-workload-socket" pod "96df7172-8d94-4726-997e-2b0c9aeb8f5d" (UID: "96df7172-8d94-4726-997e-2b0c9aeb8f5d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:14.430534 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:14.430218    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96df7172-8d94-4726-997e-2b0c9aeb8f5d-credential-socket" pod "96df7172-8d94-4726-997e-2b0c9aeb8f5d" (UID: "96df7172-8d94-4726-997e-2b0c9aeb8f5d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:14.430534 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:14.430224    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/96df7172-8d94-4726-997e-2b0c9aeb8f5d-kube-api-access-trqg9\" (UniqueName: \"kubernetes.io/projected/96df7172-8d94-4726-997e-2b0c9aeb8f5d-kube-api-access-trqg9\") pod \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\" (UID: \"96df7172-8d94-4726-997e-2b0c9aeb8f5d\") "
Oct 30 22:17:14.430534 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:14.430504    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96df7172-8d94-4726-997e-2b0c9aeb8f5d-istiod-ca-cert" pod "96df7172-8d94-4726-997e-2b0c9aeb8f5d" (UID: "96df7172-8d94-4726-997e-2b0c9aeb8f5d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:17:14.430534 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:14.430500    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96df7172-8d94-4726-997e-2b0c9aeb8f5d-istio-data" pod "96df7172-8d94-4726-997e-2b0c9aeb8f5d" (UID: "96df7172-8d94-4726-997e-2b0c9aeb8f5d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:14.430912 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:14.430588    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96df7172-8d94-4726-997e-2b0c9aeb8f5d-workload-certs" pod "96df7172-8d94-4726-997e-2b0c9aeb8f5d" (UID: "96df7172-8d94-4726-997e-2b0c9aeb8f5d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:14.430912 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:14.430878    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96df7172-8d94-4726-997e-2b0c9aeb8f5d-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:14.430912 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:14.430900    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96df7172-8d94-4726-997e-2b0c9aeb8f5d-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:14.431056 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:14.430915    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96df7172-8d94-4726-997e-2b0c9aeb8f5d-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:14.431056 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:14.430930    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96df7172-8d94-4726-997e-2b0c9aeb8f5d-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:14.431056 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:14.430946    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96df7172-8d94-4726-997e-2b0c9aeb8f5d-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:14.432560 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:14.432529    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96df7172-8d94-4726-997e-2b0c9aeb8f5d-istio-token" pod "96df7172-8d94-4726-997e-2b0c9aeb8f5d" (UID: "96df7172-8d94-4726-997e-2b0c9aeb8f5d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:14.432693 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:14.432591    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96df7172-8d94-4726-997e-2b0c9aeb8f5d-istio-envoy" pod "96df7172-8d94-4726-997e-2b0c9aeb8f5d" (UID: "96df7172-8d94-4726-997e-2b0c9aeb8f5d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:14.432823 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:14.432805    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96df7172-8d94-4726-997e-2b0c9aeb8f5d-kube-api-access-trqg9" pod "96df7172-8d94-4726-997e-2b0c9aeb8f5d" (UID: "96df7172-8d94-4726-997e-2b0c9aeb8f5d"). InnerVolumeSpecName "kube-api-access-trqg9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:14.433002 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:14.432976    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/96df7172-8d94-4726-997e-2b0c9aeb8f5d-istio-podinfo" pod "96df7172-8d94-4726-997e-2b0c9aeb8f5d" (UID: "96df7172-8d94-4726-997e-2b0c9aeb8f5d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:17:14.532069 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:14.532025    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96df7172-8d94-4726-997e-2b0c9aeb8f5d-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:14.532069 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:14.532071    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trqg9\" (UniqueName: \"kubernetes.io/projected/96df7172-8d94-4726-997e-2b0c9aeb8f5d-kube-api-access-trqg9\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:14.532365 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:14.532087    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96df7172-8d94-4726-997e-2b0c9aeb8f5d-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:14.532365 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:14.532101    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96df7172-8d94-4726-997e-2b0c9aeb8f5d-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:17:14.869968 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:14.869931    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:15.022684 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:15.022648    4871 generic.go:356] "Generic (PLEG): container finished" podID="96df7172-8d94-4726-997e-2b0c9aeb8f5d" containerID="18a1b8d2852d239fe6b9af99f128bd5c89bee4de0808f52bac955f5f9bea7a63" exitCode=0
Oct 30 22:17:15.022892 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:15.022701    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"
Oct 30 22:17:15.022892 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:15.022737    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7" event={"ID":"96df7172-8d94-4726-997e-2b0c9aeb8f5d","Type":"ContainerDied","Data":"18a1b8d2852d239fe6b9af99f128bd5c89bee4de0808f52bac955f5f9bea7a63"}
Oct 30 22:17:15.022892 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:15.022807    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7" event={"ID":"96df7172-8d94-4726-997e-2b0c9aeb8f5d","Type":"ContainerDied","Data":"3fb5b372ab24600b7d9d4261ee001801e36106225e821d7f72805012780ef5d3"}
Oct 30 22:17:15.022892 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:15.022839    4871 scope.go:122] "RemoveContainer" containerID="18a1b8d2852d239fe6b9af99f128bd5c89bee4de0808f52bac955f5f9bea7a63"
Oct 30 22:17:15.036567 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:15.036545    4871 scope.go:122] "RemoveContainer" containerID="7d568e31411f948f4cf9680acd726967cd4fb7ebbcb98db95476ec161f5e201c"
Oct 30 22:17:15.040753 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:15.040721    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:15.048103 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:15.048081    4871 scope.go:122] "RemoveContainer" containerID="18a1b8d2852d239fe6b9af99f128bd5c89bee4de0808f52bac955f5f9bea7a63"
Oct 30 22:17:15.048449 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:17:15.048423    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18a1b8d2852d239fe6b9af99f128bd5c89bee4de0808f52bac955f5f9bea7a63\": container with ID starting with 18a1b8d2852d239fe6b9af99f128bd5c89bee4de0808f52bac955f5f9bea7a63 not found: ID does not exist" containerID="18a1b8d2852d239fe6b9af99f128bd5c89bee4de0808f52bac955f5f9bea7a63"
Oct 30 22:17:15.048517 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:15.048457    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18a1b8d2852d239fe6b9af99f128bd5c89bee4de0808f52bac955f5f9bea7a63"} err="failed to get container status \"18a1b8d2852d239fe6b9af99f128bd5c89bee4de0808f52bac955f5f9bea7a63\": rpc error: code = NotFound desc = could not find container \"18a1b8d2852d239fe6b9af99f128bd5c89bee4de0808f52bac955f5f9bea7a63\": container with ID starting with 18a1b8d2852d239fe6b9af99f128bd5c89bee4de0808f52bac955f5f9bea7a63 not found: ID does not exist"
Oct 30 22:17:15.048517 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:15.048477    4871 scope.go:122] "RemoveContainer" containerID="7d568e31411f948f4cf9680acd726967cd4fb7ebbcb98db95476ec161f5e201c"
Oct 30 22:17:15.048822 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:17:15.048797    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d568e31411f948f4cf9680acd726967cd4fb7ebbcb98db95476ec161f5e201c\": container with ID starting with 7d568e31411f948f4cf9680acd726967cd4fb7ebbcb98db95476ec161f5e201c not found: ID does not exist" containerID="7d568e31411f948f4cf9680acd726967cd4fb7ebbcb98db95476ec161f5e201c"
Oct 30 22:17:15.048926 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:15.048826    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d568e31411f948f4cf9680acd726967cd4fb7ebbcb98db95476ec161f5e201c"} err="failed to get container status \"7d568e31411f948f4cf9680acd726967cd4fb7ebbcb98db95476ec161f5e201c\": rpc error: code = NotFound desc = could not find container \"7d568e31411f948f4cf9680acd726967cd4fb7ebbcb98db95476ec161f5e201c\": container with ID starting with 7d568e31411f948f4cf9680acd726967cd4fb7ebbcb98db95476ec161f5e201c not found: ID does not exist"
Oct 30 22:17:15.200781 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:15.200666    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:15.254865 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:15.254824    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"]
Oct 30 22:17:15.294190 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:15.294149    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-a3aa65-openshift-default-57649d9c-657f7"]
Oct 30 22:17:15.961642 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:15.961607    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="96df7172-8d94-4726-997e-2b0c9aeb8f5d" path="/var/lib/kubelet/pods/96df7172-8d94-4726-997e-2b0c9aeb8f5d/volumes"
Oct 30 22:17:18.120518 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:18.120436    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:18.324482 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:18.324444    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:18.725362 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:18.715038    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-63a282-5978dfffdf-wtlnf"]
Oct 30 22:17:18.725362 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:18.718261    4871 cpu_manager.go:414] "containerMap: removing container" podUID="96df7172-8d94-4726-997e-2b0c9aeb8f5d" containerName="istio-proxy"
Oct 30 22:17:18.725362 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:18.718292    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="96df7172-8d94-4726-997e-2b0c9aeb8f5d" containerName="istio-proxy"
Oct 30 22:17:18.725362 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:18.720157    4871 cpu_manager.go:414] "containerMap: removing container" podUID="96df7172-8d94-4726-997e-2b0c9aeb8f5d" containerName="istio-proxy"
Oct 30 22:17:18.725362 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:18.720185    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="96df7172-8d94-4726-997e-2b0c9aeb8f5d" containerName="istio-proxy"
Oct 30 22:17:18.725362 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:18.720467    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="96df7172-8d94-4726-997e-2b0c9aeb8f5d" containerName="istio-proxy"
Oct 30 22:17:18.725362 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:18.720484    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="96df7172-8d94-4726-997e-2b0c9aeb8f5d" containerName="istio-proxy"
Oct 30 22:17:18.728274 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:18.726911    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-63a282-5978dfffdf-wtlnf"
Oct 30 22:17:18.768041 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:18.768005    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-63a282-5978dfffdf-wtlnf"]
Oct 30 22:17:18.803258 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:18.798696    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-63a282-dockercfg-78vcw\""
Oct 30 22:17:18.803258 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:18.798799    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-63a282\""
Oct 30 22:17:18.845706 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:18.845668    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd380e6f-9f62-45c9-ba8c-a4b16e749436-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-63a282-5978dfffdf-wtlnf\" (UID: \"dd380e6f-9f62-45c9-ba8c-a4b16e749436\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-63a282-5978dfffdf-wtlnf"
Oct 30 22:17:18.846016 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:18.846003    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2p6ww\" (UniqueName: \"kubernetes.io/projected/dd380e6f-9f62-45c9-ba8c-a4b16e749436-kube-api-access-2p6ww\") pod \"payload-processing-e2e-ipp-a-63a282-5978dfffdf-wtlnf\" (UID: \"dd380e6f-9f62-45c9-ba8c-a4b16e749436\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-63a282-5978dfffdf-wtlnf"
Oct 30 22:17:18.873349 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:18.873306    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-63a282-6c49469979-hf22h"]
Oct 30 22:17:18.880749 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:18.880714    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-63a282-6c49469979-hf22h"
Oct 30 22:17:18.921946 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:18.921900    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-63a282-6c49469979-hf22h"]
Oct 30 22:17:18.947108 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:18.947063    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2p6ww\" (UniqueName: \"kubernetes.io/projected/dd380e6f-9f62-45c9-ba8c-a4b16e749436-kube-api-access-2p6ww\") pod \"payload-processing-e2e-ipp-a-63a282-5978dfffdf-wtlnf\" (UID: \"dd380e6f-9f62-45c9-ba8c-a4b16e749436\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-63a282-5978dfffdf-wtlnf"
Oct 30 22:17:18.947352 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:18.947221    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c399f24-ffaf-4649-89c9-49de27e94c41-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-63a282-6c49469979-hf22h\" (UID: \"8c399f24-ffaf-4649-89c9-49de27e94c41\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-63a282-6c49469979-hf22h"
Oct 30 22:17:18.947439 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:18.947411    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd380e6f-9f62-45c9-ba8c-a4b16e749436-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-63a282-5978dfffdf-wtlnf\" (UID: \"dd380e6f-9f62-45c9-ba8c-a4b16e749436\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-63a282-5978dfffdf-wtlnf"
Oct 30 22:17:18.947521 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:18.947482    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7hbm\" (UniqueName: \"kubernetes.io/projected/8c399f24-ffaf-4649-89c9-49de27e94c41-kube-api-access-k7hbm\") pod \"payload-pre-processing-e2e-ipp-a-63a282-6c49469979-hf22h\" (UID: \"8c399f24-ffaf-4649-89c9-49de27e94c41\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-63a282-6c49469979-hf22h"
Oct 30 22:17:18.948625 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:18.948598    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd380e6f-9f62-45c9-ba8c-a4b16e749436-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-63a282-5978dfffdf-wtlnf\" (UID: \"dd380e6f-9f62-45c9-ba8c-a4b16e749436\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-63a282-5978dfffdf-wtlnf"
Oct 30 22:17:19.007355 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:19.007282    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2p6ww\" (UniqueName: \"kubernetes.io/projected/dd380e6f-9f62-45c9-ba8c-a4b16e749436-kube-api-access-2p6ww\") pod \"payload-processing-e2e-ipp-a-63a282-5978dfffdf-wtlnf\" (UID: \"dd380e6f-9f62-45c9-ba8c-a4b16e749436\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-63a282-5978dfffdf-wtlnf"
Oct 30 22:17:19.049139 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:19.049092    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c399f24-ffaf-4649-89c9-49de27e94c41-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-63a282-6c49469979-hf22h\" (UID: \"8c399f24-ffaf-4649-89c9-49de27e94c41\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-63a282-6c49469979-hf22h"
Oct 30 22:17:19.049386 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:19.049205    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7hbm\" (UniqueName: \"kubernetes.io/projected/8c399f24-ffaf-4649-89c9-49de27e94c41-kube-api-access-k7hbm\") pod \"payload-pre-processing-e2e-ipp-a-63a282-6c49469979-hf22h\" (UID: \"8c399f24-ffaf-4649-89c9-49de27e94c41\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-63a282-6c49469979-hf22h"
Oct 30 22:17:19.050173 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:19.050150    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c399f24-ffaf-4649-89c9-49de27e94c41-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-63a282-6c49469979-hf22h\" (UID: \"8c399f24-ffaf-4649-89c9-49de27e94c41\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-63a282-6c49469979-hf22h"
Oct 30 22:17:19.076215 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:19.076174    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-63a282-5978dfffdf-wtlnf"
Oct 30 22:17:19.109290 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:19.109227    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7hbm\" (UniqueName: \"kubernetes.io/projected/8c399f24-ffaf-4649-89c9-49de27e94c41-kube-api-access-k7hbm\") pod \"payload-pre-processing-e2e-ipp-a-63a282-6c49469979-hf22h\" (UID: \"8c399f24-ffaf-4649-89c9-49de27e94c41\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-63a282-6c49469979-hf22h"
Oct 30 22:17:19.197229 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:19.197190    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-63a282-6c49469979-hf22h"
Oct 30 22:17:19.460782 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:19.460743    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-63a282-5978dfffdf-wtlnf"]
Oct 30 22:17:19.461176 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:17:19.461145    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd380e6f_9f62_45c9_ba8c_a4b16e749436.slice/crio-96f1865ded9fa5d91c97260b91d7a34c37579f7ae4b016c87ee219e78ed8b56c WatchSource:0}: Error finding container 96f1865ded9fa5d91c97260b91d7a34c37579f7ae4b016c87ee219e78ed8b56c: Status 404 returned error can't find the container with id 96f1865ded9fa5d91c97260b91d7a34c37579f7ae4b016c87ee219e78ed8b56c
Oct 30 22:17:19.587727 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:19.587695    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-63a282-6c49469979-hf22h"]
Oct 30 22:17:19.587873 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:17:19.587743    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c399f24_ffaf_4649_89c9_49de27e94c41.slice/crio-ae9c956f5f167a4eb409e116d807e87375aa917afba57dec2020c46bcb71e2fc WatchSource:0}: Error finding container ae9c956f5f167a4eb409e116d807e87375aa917afba57dec2020c46bcb71e2fc: Status 404 returned error can't find the container with id ae9c956f5f167a4eb409e116d807e87375aa917afba57dec2020c46bcb71e2fc
Oct 30 22:17:20.064735 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:20.064689    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-63a282-5978dfffdf-wtlnf" event={"ID":"dd380e6f-9f62-45c9-ba8c-a4b16e749436","Type":"ContainerStarted","Data":"1c136abc1d059a566e10da3a2bdb4aa05393f86ffdc7d2a145458fafb891f70d"}
Oct 30 22:17:20.064947 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:20.064770    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-63a282-5978dfffdf-wtlnf" event={"ID":"dd380e6f-9f62-45c9-ba8c-a4b16e749436","Type":"ContainerStarted","Data":"96f1865ded9fa5d91c97260b91d7a34c37579f7ae4b016c87ee219e78ed8b56c"}
Oct 30 22:17:20.064947 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:20.064912    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-63a282-5978dfffdf-wtlnf"
Oct 30 22:17:20.066587 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:20.066558    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-63a282-6c49469979-hf22h" event={"ID":"8c399f24-ffaf-4649-89c9-49de27e94c41","Type":"ContainerStarted","Data":"af40da7fc0e7cab19c905ede21a0b0036b271d03d51cf74197e840e2e37f06e0"}
Oct 30 22:17:20.066730 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:20.066604    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-63a282-6c49469979-hf22h" event={"ID":"8c399f24-ffaf-4649-89c9-49de27e94c41","Type":"ContainerStarted","Data":"ae9c956f5f167a4eb409e116d807e87375aa917afba57dec2020c46bcb71e2fc"}
Oct 30 22:17:20.066730 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:20.066646    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-63a282-6c49469979-hf22h"
Oct 30 22:17:20.178119 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:20.178051    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-63a282-5978dfffdf-wtlnf" podStartSLOduration=2.178034401 podStartE2EDuration="2.178034401s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:17:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:17:20.174976413 +0000 UTC m=+2382.352465029" watchObservedRunningTime="2026-10-30 22:17:20.178034401 +0000 UTC m=+2382.355523017"
Oct 30 22:17:20.289302 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:20.289194    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-63a282-6c49469979-hf22h" podStartSLOduration=2.289177878 podStartE2EDuration="2.289177878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:17:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:17:20.285406163 +0000 UTC m=+2382.462894782" watchObservedRunningTime="2026-10-30 22:17:20.289177878 +0000 UTC m=+2382.466666495"
Oct 30 22:17:20.338115 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:20.338023    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:25.102385 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:25.102348    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:30.747769 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:30.747729    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:30.896996 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:30.896953    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:31.077479 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:31.077390    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-63a282-5978dfffdf-wtlnf"
Oct 30 22:17:31.077674 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:31.077658    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-63a282-6c49469979-hf22h"
Oct 30 22:17:31.394646 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:31.394613    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:34.766558 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:34.766513    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:35.231934 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:35.231894    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:35.570722 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:17:35.570632    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:17:36.340814 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:36.340774    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-138566-6ffd477879-wbs5f"]
Oct 30 22:17:36.345181 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:36.345161    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-138566-6ffd477879-wbs5f"
Oct 30 22:17:36.399280 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:36.399228    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-138566-6ffd477879-wbs5f"]
Oct 30 22:17:36.415711 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:36.415665    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5937031a-6886-437b-98e2-6db098950b8c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-138566-6ffd477879-wbs5f\" (UID: \"5937031a-6886-437b-98e2-6db098950b8c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-138566-6ffd477879-wbs5f"
Oct 30 22:17:36.415918 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:36.415728    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9vnp\" (UniqueName: \"kubernetes.io/projected/5937031a-6886-437b-98e2-6db098950b8c-kube-api-access-q9vnp\") pod \"payload-pre-processing-e2e-ipp-b-138566-6ffd477879-wbs5f\" (UID: \"5937031a-6886-437b-98e2-6db098950b8c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-138566-6ffd477879-wbs5f"
Oct 30 22:17:36.441996 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:36.441946    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-138566\""
Oct 30 22:17:36.441996 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:36.441966    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-138566-dockercfg-9d8ld\""
Oct 30 22:17:36.517408 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:36.517359    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5937031a-6886-437b-98e2-6db098950b8c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-138566-6ffd477879-wbs5f\" (UID: \"5937031a-6886-437b-98e2-6db098950b8c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-138566-6ffd477879-wbs5f"
Oct 30 22:17:36.517628 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:36.517467    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9vnp\" (UniqueName: \"kubernetes.io/projected/5937031a-6886-437b-98e2-6db098950b8c-kube-api-access-q9vnp\") pod \"payload-pre-processing-e2e-ipp-b-138566-6ffd477879-wbs5f\" (UID: \"5937031a-6886-437b-98e2-6db098950b8c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-138566-6ffd477879-wbs5f"
Oct 30 22:17:36.518128 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:36.518096    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5937031a-6886-437b-98e2-6db098950b8c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-138566-6ffd477879-wbs5f\" (UID: \"5937031a-6886-437b-98e2-6db098950b8c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-138566-6ffd477879-wbs5f"
Oct 30 22:17:36.576730 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:36.576698    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9vnp\" (UniqueName: \"kubernetes.io/projected/5937031a-6886-437b-98e2-6db098950b8c-kube-api-access-q9vnp\") pod \"payload-pre-processing-e2e-ipp-b-138566-6ffd477879-wbs5f\" (UID: \"5937031a-6886-437b-98e2-6db098950b8c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-138566-6ffd477879-wbs5f"
Oct 30 22:17:36.660503 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:36.660464    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-138566-6ffd477879-wbs5f"
Oct 30 22:17:37.052831 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:37.052802    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-138566-6ffd477879-wbs5f"]
Oct 30 22:17:37.053821 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:17:37.053788    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5937031a_6886_437b_98e2_6db098950b8c.slice/crio-d371f89bb94959f817f33486b7ce7c1200c4b5d19cf988f701a046080ceace8d WatchSource:0}: Error finding container d371f89bb94959f817f33486b7ce7c1200c4b5d19cf988f701a046080ceace8d: Status 404 returned error can't find the container with id d371f89bb94959f817f33486b7ce7c1200c4b5d19cf988f701a046080ceace8d
Oct 30 22:17:37.208666 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:37.208614    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-138566-6ffd477879-wbs5f" event={"ID":"5937031a-6886-437b-98e2-6db098950b8c","Type":"ContainerStarted","Data":"8d66dc380b46e446b35b11ec3b92d3fd98c4022cf3cb012dcf255a1c8f226866"}
Oct 30 22:17:37.208852 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:37.208683    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-138566-6ffd477879-wbs5f" event={"ID":"5937031a-6886-437b-98e2-6db098950b8c","Type":"ContainerStarted","Data":"d371f89bb94959f817f33486b7ce7c1200c4b5d19cf988f701a046080ceace8d"}
Oct 30 22:17:37.208852 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:37.208726    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-138566-6ffd477879-wbs5f"
Oct 30 22:17:37.321144 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:37.321018    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-138566-6ffd477879-wbs5f" podStartSLOduration=1.321002228 podStartE2EDuration="1.321002228s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:17:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:17:37.319152325 +0000 UTC m=+2399.496640945" watchObservedRunningTime="2026-10-30 22:17:37.321002228 +0000 UTC m=+2399.498490845"
Oct 30 22:17:37.805555 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:37.805503    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:38.437272 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:38.437206    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx_8d401654-8a4c-4a79-b2d3-612765312421/extract/0.log"
Oct 30 22:17:38.438676 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:38.438645    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx_8d401654-8a4c-4a79-b2d3-612765312421/extract/0.log"
Oct 30 22:17:38.451295 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:38.451231    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/2.log"
Oct 30 22:17:38.452692 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:38.452665    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/2.log"
Oct 30 22:17:38.458663 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:38.458640    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzntd_2360fddc-52a5-42a5-82ca-a05aff84d217/ovn-acl-logging/0.log"
Oct 30 22:17:38.459808 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:38.459790    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzntd_2360fddc-52a5-42a5-82ca-a05aff84d217/ovn-acl-logging/0.log"
Oct 30 22:17:38.929752 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:38.929708    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:39.105659 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:39.105622    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:42.272125 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:42.272095    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:42.420639 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:42.420595    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:47.252468 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:47.252435    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:48.179787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:48.179752    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:48.223195 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:48.223162    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-138566-6ffd477879-wbs5f"
Oct 30 22:17:54.259556 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:54.259510    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:57.664737 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:57.664696    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:58.134366 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:58.134314    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:58.786904 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:58.786866    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-fe745d-85cf688ff4-xh6cf"]
Oct 30 22:17:58.793127 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:58.793101    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-fe745d-85cf688ff4-xh6cf"
Oct 30 22:17:58.820692 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:58.820659    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rvwt\" (UniqueName: \"kubernetes.io/projected/baab5eee-6bad-4516-9084-049003c4f2d3-kube-api-access-4rvwt\") pod \"payload-processing-e2e-ipp-b-fe745d-85cf688ff4-xh6cf\" (UID: \"baab5eee-6bad-4516-9084-049003c4f2d3\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-fe745d-85cf688ff4-xh6cf"
Oct 30 22:17:58.820855 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:58.820799    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/baab5eee-6bad-4516-9084-049003c4f2d3-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-fe745d-85cf688ff4-xh6cf\" (UID: \"baab5eee-6bad-4516-9084-049003c4f2d3\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-fe745d-85cf688ff4-xh6cf"
Oct 30 22:17:58.847044 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:58.847009    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-fe745d-85cf688ff4-xh6cf"]
Oct 30 22:17:58.888884 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:58.888848    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-fe745d-dockercfg-vb2kk\""
Oct 30 22:17:58.888884 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:58.888857    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-fe745d\""
Oct 30 22:17:58.922065 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:58.922019    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rvwt\" (UniqueName: \"kubernetes.io/projected/baab5eee-6bad-4516-9084-049003c4f2d3-kube-api-access-4rvwt\") pod \"payload-processing-e2e-ipp-b-fe745d-85cf688ff4-xh6cf\" (UID: \"baab5eee-6bad-4516-9084-049003c4f2d3\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-fe745d-85cf688ff4-xh6cf"
Oct 30 22:17:58.922322 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:58.922171    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/baab5eee-6bad-4516-9084-049003c4f2d3-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-fe745d-85cf688ff4-xh6cf\" (UID: \"baab5eee-6bad-4516-9084-049003c4f2d3\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-fe745d-85cf688ff4-xh6cf"
Oct 30 22:17:58.922969 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:58.922948    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/baab5eee-6bad-4516-9084-049003c4f2d3-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-fe745d-85cf688ff4-xh6cf\" (UID: \"baab5eee-6bad-4516-9084-049003c4f2d3\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-fe745d-85cf688ff4-xh6cf"
Oct 30 22:17:58.980535 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:58.980498    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rvwt\" (UniqueName: \"kubernetes.io/projected/baab5eee-6bad-4516-9084-049003c4f2d3-kube-api-access-4rvwt\") pod \"payload-processing-e2e-ipp-b-fe745d-85cf688ff4-xh6cf\" (UID: \"baab5eee-6bad-4516-9084-049003c4f2d3\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-fe745d-85cf688ff4-xh6cf"
Oct 30 22:17:59.108119 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:59.108022    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-fe745d-85cf688ff4-xh6cf"
Oct 30 22:17:59.488275 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:17:59.486477    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-fe745d-85cf688ff4-xh6cf"]
Oct 30 22:18:00.199514 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:00.199474    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:00.429751 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:00.429704    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-fe745d-85cf688ff4-xh6cf" event={"ID":"baab5eee-6bad-4516-9084-049003c4f2d3","Type":"ContainerStarted","Data":"11e191ed2c0e8f74506405dd4e687b4db3b7649bcdd948dad2b078e71041e59a"}
Oct 30 22:18:00.429957 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:00.429777    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-fe745d-85cf688ff4-xh6cf"
Oct 30 22:18:00.429957 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:00.429801    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-fe745d-85cf688ff4-xh6cf" event={"ID":"baab5eee-6bad-4516-9084-049003c4f2d3","Type":"ContainerStarted","Data":"40cf307e8869fa18075084c7d594b8680707bb6580fadb8eff1923e38dd9d4c2"}
Oct 30 22:18:00.546139 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:00.546016    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-fe745d-85cf688ff4-xh6cf" podStartSLOduration=2.545997186 podStartE2EDuration="2.545997186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:17:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:00.542038561 +0000 UTC m=+2422.719527178" watchObservedRunningTime="2026-10-30 22:18:00.545997186 +0000 UTC m=+2422.723485806"
Oct 30 22:18:00.619618 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:00.619573    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:05.221886 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:05.221848    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:05.568892 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:18:05.568806    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:18:09.672590 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:09.672550    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf"]
Oct 30 22:18:09.673054 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:09.672781    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf" podUID="9d801bc6-50a2-4eeb-9038-f3758bf7bf6a" containerName="maas-api" containerID="cri-o://87bf0ba967e77ec9520e5389c924d571dd5cd7dcace054cd72e08f72edd993a1" gracePeriod=30
Oct 30 22:18:09.781119 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:09.781081    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:10.007604 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.007569    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf"
Oct 30 22:18:10.075806 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.075767    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9d801bc6-50a2-4eeb-9038-f3758bf7bf6a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9d801bc6-50a2-4eeb-9038-f3758bf7bf6a-maas-api-tls\") pod \"9d801bc6-50a2-4eeb-9038-f3758bf7bf6a\" (UID: \"9d801bc6-50a2-4eeb-9038-f3758bf7bf6a\") "
Oct 30 22:18:10.076012 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.075856    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9d801bc6-50a2-4eeb-9038-f3758bf7bf6a-kube-api-access-4qwjx\" (UniqueName: \"kubernetes.io/projected/9d801bc6-50a2-4eeb-9038-f3758bf7bf6a-kube-api-access-4qwjx\") pod \"9d801bc6-50a2-4eeb-9038-f3758bf7bf6a\" (UID: \"9d801bc6-50a2-4eeb-9038-f3758bf7bf6a\") "
Oct 30 22:18:10.076012 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.075914    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9d801bc6-50a2-4eeb-9038-f3758bf7bf6a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d801bc6-50a2-4eeb-9038-f3758bf7bf6a-openshift-service-ca\") pod \"9d801bc6-50a2-4eeb-9038-f3758bf7bf6a\" (UID: \"9d801bc6-50a2-4eeb-9038-f3758bf7bf6a\") "
Oct 30 22:18:10.076166 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.076138    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9d801bc6-50a2-4eeb-9038-f3758bf7bf6a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d801bc6-50a2-4eeb-9038-f3758bf7bf6a-metrics-certs\") pod \"9d801bc6-50a2-4eeb-9038-f3758bf7bf6a\" (UID: \"9d801bc6-50a2-4eeb-9038-f3758bf7bf6a\") "
Oct 30 22:18:10.076417 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.076387    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d801bc6-50a2-4eeb-9038-f3758bf7bf6a-openshift-service-ca" pod "9d801bc6-50a2-4eeb-9038-f3758bf7bf6a" (UID: "9d801bc6-50a2-4eeb-9038-f3758bf7bf6a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:18:10.077539 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.077508    4871 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d801bc6-50a2-4eeb-9038-f3758bf7bf6a-openshift-service-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:10.083345 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.083308    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d801bc6-50a2-4eeb-9038-f3758bf7bf6a-maas-api-tls" pod "9d801bc6-50a2-4eeb-9038-f3758bf7bf6a" (UID: "9d801bc6-50a2-4eeb-9038-f3758bf7bf6a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:18:10.084805 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.084767    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d801bc6-50a2-4eeb-9038-f3758bf7bf6a-kube-api-access-4qwjx" pod "9d801bc6-50a2-4eeb-9038-f3758bf7bf6a" (UID: "9d801bc6-50a2-4eeb-9038-f3758bf7bf6a"). InnerVolumeSpecName "kube-api-access-4qwjx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:18:10.091651 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.091615    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d801bc6-50a2-4eeb-9038-f3758bf7bf6a-metrics-certs" pod "9d801bc6-50a2-4eeb-9038-f3758bf7bf6a" (UID: "9d801bc6-50a2-4eeb-9038-f3758bf7bf6a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:18:10.113199 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.113168    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a4257d-c6b8d878d-kr4t7"]
Oct 30 22:18:10.113478 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.113428    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-a4257d-c6b8d878d-kr4t7" podUID="7d8bd465-4a7c-4480-9079-eac8aed7c7a2" containerName="payload-processing" containerID="cri-o://7cbc91cf4acb3c900deedec0372df35742ee58e3904f2f4d0615c2112e7fa2a7" gracePeriod=30
Oct 30 22:18:10.179039 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.179001    4871 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d801bc6-50a2-4eeb-9038-f3758bf7bf6a-metrics-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:10.179039 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.179036    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9d801bc6-50a2-4eeb-9038-f3758bf7bf6a-maas-api-tls\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:10.179254 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.179048    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qwjx\" (UniqueName: \"kubernetes.io/projected/9d801bc6-50a2-4eeb-9038-f3758bf7bf6a-kube-api-access-4qwjx\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:10.291895 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.291848    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:10.374788 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.374751    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-a4257d-c6b8d878d-kr4t7"
Oct 30 22:18:10.450066 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.450033    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:10.482465 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.482380    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7d8bd465-4a7c-4480-9079-eac8aed7c7a2-kube-api-access-5hp9t\" (UniqueName: \"kubernetes.io/projected/7d8bd465-4a7c-4480-9079-eac8aed7c7a2-kube-api-access-5hp9t\") pod \"7d8bd465-4a7c-4480-9079-eac8aed7c7a2\" (UID: \"7d8bd465-4a7c-4480-9079-eac8aed7c7a2\") "
Oct 30 22:18:10.482650 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.482486    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7d8bd465-4a7c-4480-9079-eac8aed7c7a2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d8bd465-4a7c-4480-9079-eac8aed7c7a2-plugins-config-volume\") pod \"7d8bd465-4a7c-4480-9079-eac8aed7c7a2\" (UID: \"7d8bd465-4a7c-4480-9079-eac8aed7c7a2\") "
Oct 30 22:18:10.482915 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.482890    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d8bd465-4a7c-4480-9079-eac8aed7c7a2-plugins-config-volume" pod "7d8bd465-4a7c-4480-9079-eac8aed7c7a2" (UID: "7d8bd465-4a7c-4480-9079-eac8aed7c7a2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:18:10.483088 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.483069    4871 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d8bd465-4a7c-4480-9079-eac8aed7c7a2-plugins-config-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:10.484606 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.484584    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d8bd465-4a7c-4480-9079-eac8aed7c7a2-kube-api-access-5hp9t" pod "7d8bd465-4a7c-4480-9079-eac8aed7c7a2" (UID: "7d8bd465-4a7c-4480-9079-eac8aed7c7a2"). InnerVolumeSpecName "kube-api-access-5hp9t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:18:10.504836 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.504799    4871 generic.go:356] "Generic (PLEG): container finished" podID="7d8bd465-4a7c-4480-9079-eac8aed7c7a2" containerID="7cbc91cf4acb3c900deedec0372df35742ee58e3904f2f4d0615c2112e7fa2a7" exitCode=0
Oct 30 22:18:10.505021 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.504848    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-a4257d-c6b8d878d-kr4t7"
Oct 30 22:18:10.505021 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.504880    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a4257d-c6b8d878d-kr4t7" event={"ID":"7d8bd465-4a7c-4480-9079-eac8aed7c7a2","Type":"ContainerDied","Data":"7cbc91cf4acb3c900deedec0372df35742ee58e3904f2f4d0615c2112e7fa2a7"}
Oct 30 22:18:10.505021 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.504934    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a4257d-c6b8d878d-kr4t7" event={"ID":"7d8bd465-4a7c-4480-9079-eac8aed7c7a2","Type":"ContainerDied","Data":"733aafcfe807959f771314c686de588cb52ed02c0c46c59ff1380a7f039729a0"}
Oct 30 22:18:10.505021 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.504948    4871 scope.go:122] "RemoveContainer" containerID="7cbc91cf4acb3c900deedec0372df35742ee58e3904f2f4d0615c2112e7fa2a7"
Oct 30 22:18:10.507051 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.507021    4871 generic.go:356] "Generic (PLEG): container finished" podID="9d801bc6-50a2-4eeb-9038-f3758bf7bf6a" containerID="87bf0ba967e77ec9520e5389c924d571dd5cd7dcace054cd72e08f72edd993a1" exitCode=0
Oct 30 22:18:10.507188 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.507174    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf" event={"ID":"9d801bc6-50a2-4eeb-9038-f3758bf7bf6a","Type":"ContainerDied","Data":"87bf0ba967e77ec9520e5389c924d571dd5cd7dcace054cd72e08f72edd993a1"}
Oct 30 22:18:10.507289 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.507217    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf" event={"ID":"9d801bc6-50a2-4eeb-9038-f3758bf7bf6a","Type":"ContainerDied","Data":"f9c2e13178c59a066aab9d64484451fe3e411ce3fe3b7556463ad08fa25297a9"}
Oct 30 22:18:10.507410 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.507381    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf"
Oct 30 22:18:10.522944 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.522919    4871 scope.go:122] "RemoveContainer" containerID="7cbc91cf4acb3c900deedec0372df35742ee58e3904f2f4d0615c2112e7fa2a7"
Oct 30 22:18:10.523756 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:18:10.523719    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cbc91cf4acb3c900deedec0372df35742ee58e3904f2f4d0615c2112e7fa2a7\": container with ID starting with 7cbc91cf4acb3c900deedec0372df35742ee58e3904f2f4d0615c2112e7fa2a7 not found: ID does not exist" containerID="7cbc91cf4acb3c900deedec0372df35742ee58e3904f2f4d0615c2112e7fa2a7"
Oct 30 22:18:10.523879 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.523765    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cbc91cf4acb3c900deedec0372df35742ee58e3904f2f4d0615c2112e7fa2a7"} err="failed to get container status \"7cbc91cf4acb3c900deedec0372df35742ee58e3904f2f4d0615c2112e7fa2a7\": rpc error: code = NotFound desc = could not find container \"7cbc91cf4acb3c900deedec0372df35742ee58e3904f2f4d0615c2112e7fa2a7\": container with ID starting with 7cbc91cf4acb3c900deedec0372df35742ee58e3904f2f4d0615c2112e7fa2a7 not found: ID does not exist"
Oct 30 22:18:10.523879 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.523793    4871 scope.go:122] "RemoveContainer" containerID="87bf0ba967e77ec9520e5389c924d571dd5cd7dcace054cd72e08f72edd993a1"
Oct 30 22:18:10.539895 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.539869    4871 scope.go:122] "RemoveContainer" containerID="87bf0ba967e77ec9520e5389c924d571dd5cd7dcace054cd72e08f72edd993a1"
Oct 30 22:18:10.540307 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:18:10.540279    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87bf0ba967e77ec9520e5389c924d571dd5cd7dcace054cd72e08f72edd993a1\": container with ID starting with 87bf0ba967e77ec9520e5389c924d571dd5cd7dcace054cd72e08f72edd993a1 not found: ID does not exist" containerID="87bf0ba967e77ec9520e5389c924d571dd5cd7dcace054cd72e08f72edd993a1"
Oct 30 22:18:10.540389 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.540317    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87bf0ba967e77ec9520e5389c924d571dd5cd7dcace054cd72e08f72edd993a1"} err="failed to get container status \"87bf0ba967e77ec9520e5389c924d571dd5cd7dcace054cd72e08f72edd993a1\": rpc error: code = NotFound desc = could not find container \"87bf0ba967e77ec9520e5389c924d571dd5cd7dcace054cd72e08f72edd993a1\": container with ID starting with 87bf0ba967e77ec9520e5389c924d571dd5cd7dcace054cd72e08f72edd993a1 not found: ID does not exist"
Oct 30 22:18:10.585028 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.584958    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5hp9t\" (UniqueName: \"kubernetes.io/projected/7d8bd465-4a7c-4480-9079-eac8aed7c7a2-kube-api-access-5hp9t\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:10.718184 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.718140    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf"]
Oct 30 22:18:10.727918 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.727873    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a4257d-6f6b49bb4c-fbrbf"]
Oct 30 22:18:10.826515 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.826199    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a4257d-c6b8d878d-kr4t7"]
Oct 30 22:18:10.830518 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:10.830489    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a4257d-c6b8d878d-kr4t7"]
Oct 30 22:18:11.290664 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:11.290625    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:11.440126 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:11.440097    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-fe745d-85cf688ff4-xh6cf"
Oct 30 22:18:11.580354 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:11.580226    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:11.765123 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:11.765085    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:11.962373 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:11.962342    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7d8bd465-4a7c-4480-9079-eac8aed7c7a2" path="/var/lib/kubelet/pods/7d8bd465-4a7c-4480-9079-eac8aed7c7a2/volumes"
Oct 30 22:18:11.962895 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:11.962879    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9d801bc6-50a2-4eeb-9038-f3758bf7bf6a" path="/var/lib/kubelet/pods/9d801bc6-50a2-4eeb-9038-f3758bf7bf6a/volumes"
Oct 30 22:18:12.225833 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:12.225749    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:17.863756 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:17.863723    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:18.383828 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:18.383789    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:18.576799 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:18.576753    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:19.348803 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:19.348759    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:19.518875 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:19.518835    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:19.722211 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:19.722164    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:19.728098 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:19.728058    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"]
Oct 30 22:18:19.728438 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:19.728405    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7" podUID="f9b60b37-4047-499d-87fa-2533d0e12e85" containerName="istio-proxy" containerID="cri-o://b28eac6dad69f9d340415115e0f584821e43a968cc88c5854c6d6a0bd41026bf" gracePeriod=30
Oct 30 22:18:19.930980 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:19.930843    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:20.085460 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:20.085344    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:20.243483 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:20.243433    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:20.898258 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:20.898212    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:25.005092 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.005018    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"
Oct 30 22:18:25.048191 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.048158    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f9b60b37-4047-499d-87fa-2533d0e12e85-kube-api-access-75hhs\" (UniqueName: \"kubernetes.io/projected/f9b60b37-4047-499d-87fa-2533d0e12e85-kube-api-access-75hhs\") pod \"f9b60b37-4047-499d-87fa-2533d0e12e85\" (UID: \"f9b60b37-4047-499d-87fa-2533d0e12e85\") "
Oct 30 22:18:25.048406 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.048267    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9b60b37-4047-499d-87fa-2533d0e12e85-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9b60b37-4047-499d-87fa-2533d0e12e85-workload-socket\") pod \"f9b60b37-4047-499d-87fa-2533d0e12e85\" (UID: \"f9b60b37-4047-499d-87fa-2533d0e12e85\") "
Oct 30 22:18:25.048406 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.048309    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f9b60b37-4047-499d-87fa-2533d0e12e85-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9b60b37-4047-499d-87fa-2533d0e12e85-istio-podinfo\") pod \"f9b60b37-4047-499d-87fa-2533d0e12e85\" (UID: \"f9b60b37-4047-499d-87fa-2533d0e12e85\") "
Oct 30 22:18:25.048406 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.048373    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9b60b37-4047-499d-87fa-2533d0e12e85-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9b60b37-4047-499d-87fa-2533d0e12e85-credential-socket\") pod \"f9b60b37-4047-499d-87fa-2533d0e12e85\" (UID: \"f9b60b37-4047-499d-87fa-2533d0e12e85\") "
Oct 30 22:18:25.048406 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.048393    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f9b60b37-4047-499d-87fa-2533d0e12e85-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9b60b37-4047-499d-87fa-2533d0e12e85-istiod-ca-cert\") pod \"f9b60b37-4047-499d-87fa-2533d0e12e85\" (UID: \"f9b60b37-4047-499d-87fa-2533d0e12e85\") "
Oct 30 22:18:25.048625 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.048411    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9b60b37-4047-499d-87fa-2533d0e12e85-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9b60b37-4047-499d-87fa-2533d0e12e85-istio-data\") pod \"f9b60b37-4047-499d-87fa-2533d0e12e85\" (UID: \"f9b60b37-4047-499d-87fa-2533d0e12e85\") "
Oct 30 22:18:25.048625 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.048474    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9b60b37-4047-499d-87fa-2533d0e12e85-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9b60b37-4047-499d-87fa-2533d0e12e85-istio-envoy\") pod \"f9b60b37-4047-499d-87fa-2533d0e12e85\" (UID: \"f9b60b37-4047-499d-87fa-2533d0e12e85\") "
Oct 30 22:18:25.048625 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.048575    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f9b60b37-4047-499d-87fa-2533d0e12e85-istio-token\" (UniqueName: \"kubernetes.io/projected/f9b60b37-4047-499d-87fa-2533d0e12e85-istio-token\") pod \"f9b60b37-4047-499d-87fa-2533d0e12e85\" (UID: \"f9b60b37-4047-499d-87fa-2533d0e12e85\") "
Oct 30 22:18:25.048625 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.048612    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9b60b37-4047-499d-87fa-2533d0e12e85-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9b60b37-4047-499d-87fa-2533d0e12e85-workload-certs\") pod \"f9b60b37-4047-499d-87fa-2533d0e12e85\" (UID: \"f9b60b37-4047-499d-87fa-2533d0e12e85\") "
Oct 30 22:18:25.048841 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.048632    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9b60b37-4047-499d-87fa-2533d0e12e85-workload-socket" pod "f9b60b37-4047-499d-87fa-2533d0e12e85" (UID: "f9b60b37-4047-499d-87fa-2533d0e12e85"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:25.048841 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.048734    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9b60b37-4047-499d-87fa-2533d0e12e85-credential-socket" pod "f9b60b37-4047-499d-87fa-2533d0e12e85" (UID: "f9b60b37-4047-499d-87fa-2533d0e12e85"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:25.048841 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.048764    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9b60b37-4047-499d-87fa-2533d0e12e85-istio-data" pod "f9b60b37-4047-499d-87fa-2533d0e12e85" (UID: "f9b60b37-4047-499d-87fa-2533d0e12e85"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:25.049113 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.049087    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9b60b37-4047-499d-87fa-2533d0e12e85-workload-certs" pod "f9b60b37-4047-499d-87fa-2533d0e12e85" (UID: "f9b60b37-4047-499d-87fa-2533d0e12e85"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:25.049248 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.049007    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9b60b37-4047-499d-87fa-2533d0e12e85-istiod-ca-cert" pod "f9b60b37-4047-499d-87fa-2533d0e12e85" (UID: "f9b60b37-4047-499d-87fa-2533d0e12e85"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:18:25.049678 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.049609    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9b60b37-4047-499d-87fa-2533d0e12e85-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:25.049678 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.049643    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9b60b37-4047-499d-87fa-2533d0e12e85-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:25.049678 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.049660    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9b60b37-4047-499d-87fa-2533d0e12e85-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:25.049678 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.049675    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9b60b37-4047-499d-87fa-2533d0e12e85-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:25.049962 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.049690    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9b60b37-4047-499d-87fa-2533d0e12e85-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:25.050956 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.050927    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9b60b37-4047-499d-87fa-2533d0e12e85-istio-envoy" pod "f9b60b37-4047-499d-87fa-2533d0e12e85" (UID: "f9b60b37-4047-499d-87fa-2533d0e12e85"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:25.051201 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.051182    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f9b60b37-4047-499d-87fa-2533d0e12e85-istio-podinfo" pod "f9b60b37-4047-499d-87fa-2533d0e12e85" (UID: "f9b60b37-4047-499d-87fa-2533d0e12e85"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:18:25.051290 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.051262    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9b60b37-4047-499d-87fa-2533d0e12e85-kube-api-access-75hhs" pod "f9b60b37-4047-499d-87fa-2533d0e12e85" (UID: "f9b60b37-4047-499d-87fa-2533d0e12e85"). InnerVolumeSpecName "kube-api-access-75hhs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:18:25.051407 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.051390    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9b60b37-4047-499d-87fa-2533d0e12e85-istio-token" pod "f9b60b37-4047-499d-87fa-2533d0e12e85" (UID: "f9b60b37-4047-499d-87fa-2533d0e12e85"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:18:25.150495 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.150458    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9b60b37-4047-499d-87fa-2533d0e12e85-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:25.150495 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.150490    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9b60b37-4047-499d-87fa-2533d0e12e85-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:25.150495 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.150500    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9b60b37-4047-499d-87fa-2533d0e12e85-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:25.150801 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.150518    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75hhs\" (UniqueName: \"kubernetes.io/projected/f9b60b37-4047-499d-87fa-2533d0e12e85-kube-api-access-75hhs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:25.629507 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.629469    4871 generic.go:356] "Generic (PLEG): container finished" podID="f9b60b37-4047-499d-87fa-2533d0e12e85" containerID="b28eac6dad69f9d340415115e0f584821e43a968cc88c5854c6d6a0bd41026bf" exitCode=0
Oct 30 22:18:25.629809 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.629527    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"
Oct 30 22:18:25.629809 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.629540    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7" event={"ID":"f9b60b37-4047-499d-87fa-2533d0e12e85","Type":"ContainerDied","Data":"b28eac6dad69f9d340415115e0f584821e43a968cc88c5854c6d6a0bd41026bf"}
Oct 30 22:18:25.629809 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.629603    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7" event={"ID":"f9b60b37-4047-499d-87fa-2533d0e12e85","Type":"ContainerDied","Data":"45f4ccfc50436e283cc8d8f1338079fff2b425d1dabadc5087e0a80972040c8e"}
Oct 30 22:18:25.629809 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.629627    4871 scope.go:122] "RemoveContainer" containerID="b28eac6dad69f9d340415115e0f584821e43a968cc88c5854c6d6a0bd41026bf"
Oct 30 22:18:25.641798 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.641774    4871 scope.go:122] "RemoveContainer" containerID="15224f9ad56a3e7cb9d01a90562ebfa96c8a99303668126beafec0be51cd51e0"
Oct 30 22:18:25.654382 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.654352    4871 scope.go:122] "RemoveContainer" containerID="b28eac6dad69f9d340415115e0f584821e43a968cc88c5854c6d6a0bd41026bf"
Oct 30 22:18:25.654833 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:18:25.654674    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b28eac6dad69f9d340415115e0f584821e43a968cc88c5854c6d6a0bd41026bf\": container with ID starting with b28eac6dad69f9d340415115e0f584821e43a968cc88c5854c6d6a0bd41026bf not found: ID does not exist" containerID="b28eac6dad69f9d340415115e0f584821e43a968cc88c5854c6d6a0bd41026bf"
Oct 30 22:18:25.654833 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.654717    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b28eac6dad69f9d340415115e0f584821e43a968cc88c5854c6d6a0bd41026bf"} err="failed to get container status \"b28eac6dad69f9d340415115e0f584821e43a968cc88c5854c6d6a0bd41026bf\": rpc error: code = NotFound desc = could not find container \"b28eac6dad69f9d340415115e0f584821e43a968cc88c5854c6d6a0bd41026bf\": container with ID starting with b28eac6dad69f9d340415115e0f584821e43a968cc88c5854c6d6a0bd41026bf not found: ID does not exist"
Oct 30 22:18:25.654833 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.654743    4871 scope.go:122] "RemoveContainer" containerID="15224f9ad56a3e7cb9d01a90562ebfa96c8a99303668126beafec0be51cd51e0"
Oct 30 22:18:25.655123 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:18:25.655093    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15224f9ad56a3e7cb9d01a90562ebfa96c8a99303668126beafec0be51cd51e0\": container with ID starting with 15224f9ad56a3e7cb9d01a90562ebfa96c8a99303668126beafec0be51cd51e0 not found: ID does not exist" containerID="15224f9ad56a3e7cb9d01a90562ebfa96c8a99303668126beafec0be51cd51e0"
Oct 30 22:18:25.655219 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.655161    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15224f9ad56a3e7cb9d01a90562ebfa96c8a99303668126beafec0be51cd51e0"} err="failed to get container status \"15224f9ad56a3e7cb9d01a90562ebfa96c8a99303668126beafec0be51cd51e0\": rpc error: code = NotFound desc = could not find container \"15224f9ad56a3e7cb9d01a90562ebfa96c8a99303668126beafec0be51cd51e0\": container with ID starting with 15224f9ad56a3e7cb9d01a90562ebfa96c8a99303668126beafec0be51cd51e0 not found: ID does not exist"
Oct 30 22:18:25.789085 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.788989    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"]
Oct 30 22:18:25.836543 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.836508    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-a4257d-openshift-default-76f5d56dc7-ctck7"]
Oct 30 22:18:25.962270 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:25.962178    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f9b60b37-4047-499d-87fa-2533d0e12e85" path="/var/lib/kubelet/pods/f9b60b37-4047-499d-87fa-2533d0e12e85/volumes"
Oct 30 22:18:26.086137 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:26.086098    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn"]
Oct 30 22:18:26.086759 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:26.086744    4871 cpu_manager.go:414] "containerMap: removing container" podUID="f9b60b37-4047-499d-87fa-2533d0e12e85" containerName="istio-proxy"
Oct 30 22:18:26.086808 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:26.086760    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f9b60b37-4047-499d-87fa-2533d0e12e85" containerName="istio-proxy"
Oct 30 22:18:26.086808 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:26.086779    4871 cpu_manager.go:414] "containerMap: removing container" podUID="9d801bc6-50a2-4eeb-9038-f3758bf7bf6a" containerName="maas-api"
Oct 30 22:18:26.086808 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:26.086785    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9d801bc6-50a2-4eeb-9038-f3758bf7bf6a" containerName="maas-api"
Oct 30 22:18:26.086927 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:26.086811    4871 cpu_manager.go:414] "containerMap: removing container" podUID="7d8bd465-4a7c-4480-9079-eac8aed7c7a2" containerName="payload-processing"
Oct 30 22:18:26.086927 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:26.086816    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7d8bd465-4a7c-4480-9079-eac8aed7c7a2" containerName="payload-processing"
Oct 30 22:18:26.086927 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:26.086912    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="f9b60b37-4047-499d-87fa-2533d0e12e85" containerName="istio-proxy"
Oct 30 22:18:26.086927 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:26.086924    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="9d801bc6-50a2-4eeb-9038-f3758bf7bf6a" containerName="maas-api"
Oct 30 22:18:26.087055 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:26.086936    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="7d8bd465-4a7c-4480-9079-eac8aed7c7a2" containerName="payload-processing"
Oct 30 22:18:26.092010 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:26.091985    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn"
Oct 30 22:18:26.142101 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:26.142059    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-63a282-serving-cert\""
Oct 30 22:18:26.147590 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:26.147552    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn"]
Oct 30 22:18:26.165963 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:26.165928    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aec98483-f8d7-4eb3-811e-5f6b86e56230-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn\" (UID: \"aec98483-f8d7-4eb3-811e-5f6b86e56230\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn"
Oct 30 22:18:26.166158 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:26.166079    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfjhs\" (UniqueName: \"kubernetes.io/projected/aec98483-f8d7-4eb3-811e-5f6b86e56230-kube-api-access-kfjhs\") pod \"maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn\" (UID: \"aec98483-f8d7-4eb3-811e-5f6b86e56230\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn"
Oct 30 22:18:26.166329 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:26.166275    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aec98483-f8d7-4eb3-811e-5f6b86e56230-maas-api-tls\") pod \"maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn\" (UID: \"aec98483-f8d7-4eb3-811e-5f6b86e56230\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn"
Oct 30 22:18:26.166489 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:26.166340    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aec98483-f8d7-4eb3-811e-5f6b86e56230-metrics-certs\") pod \"maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn\" (UID: \"aec98483-f8d7-4eb3-811e-5f6b86e56230\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn"
Oct 30 22:18:26.268017 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:26.267914    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aec98483-f8d7-4eb3-811e-5f6b86e56230-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn\" (UID: \"aec98483-f8d7-4eb3-811e-5f6b86e56230\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn"
Oct 30 22:18:26.268210 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:26.268038    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfjhs\" (UniqueName: \"kubernetes.io/projected/aec98483-f8d7-4eb3-811e-5f6b86e56230-kube-api-access-kfjhs\") pod \"maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn\" (UID: \"aec98483-f8d7-4eb3-811e-5f6b86e56230\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn"
Oct 30 22:18:26.268210 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:26.268138    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aec98483-f8d7-4eb3-811e-5f6b86e56230-maas-api-tls\") pod \"maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn\" (UID: \"aec98483-f8d7-4eb3-811e-5f6b86e56230\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn"
Oct 30 22:18:26.268210 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:26.268196    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aec98483-f8d7-4eb3-811e-5f6b86e56230-metrics-certs\") pod \"maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn\" (UID: \"aec98483-f8d7-4eb3-811e-5f6b86e56230\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn"
Oct 30 22:18:26.268409 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:18:26.268316    4871 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-63a282-serving-cert: secret "maas-api-e2e-ipp-a-63a282-serving-cert" not found
Oct 30 22:18:26.268409 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:18:26.268392    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aec98483-f8d7-4eb3-811e-5f6b86e56230-maas-api-tls podName:aec98483-f8d7-4eb3-811e-5f6b86e56230 nodeName:}" failed. No retries permitted until 2026-10-30 22:18:26.768370713 +0000 UTC m=+2448.945859330 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/aec98483-f8d7-4eb3-811e-5f6b86e56230-maas-api-tls") pod "maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn" (UID: "aec98483-f8d7-4eb3-811e-5f6b86e56230") : secret "maas-api-e2e-ipp-a-63a282-serving-cert" not found
Oct 30 22:18:26.268793 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:26.268772    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aec98483-f8d7-4eb3-811e-5f6b86e56230-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn\" (UID: \"aec98483-f8d7-4eb3-811e-5f6b86e56230\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn"
Oct 30 22:18:26.270781 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:26.270755    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aec98483-f8d7-4eb3-811e-5f6b86e56230-metrics-certs\") pod \"maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn\" (UID: \"aec98483-f8d7-4eb3-811e-5f6b86e56230\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn"
Oct 30 22:18:26.294553 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:26.294506    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:26.329515 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:26.329481    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfjhs\" (UniqueName: \"kubernetes.io/projected/aec98483-f8d7-4eb3-811e-5f6b86e56230-kube-api-access-kfjhs\") pod \"maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn\" (UID: \"aec98483-f8d7-4eb3-811e-5f6b86e56230\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn"
Oct 30 22:18:26.501511 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:26.501466    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:26.659837 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:26.659802    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:26.775018 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:26.774983    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aec98483-f8d7-4eb3-811e-5f6b86e56230-maas-api-tls\") pod \"maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn\" (UID: \"aec98483-f8d7-4eb3-811e-5f6b86e56230\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn"
Oct 30 22:18:26.777598 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:26.777570    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aec98483-f8d7-4eb3-811e-5f6b86e56230-maas-api-tls\") pod \"maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn\" (UID: \"aec98483-f8d7-4eb3-811e-5f6b86e56230\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn"
Oct 30 22:18:27.007225 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:27.007131    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn"
Oct 30 22:18:27.220560 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:27.220514    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:27.378853 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:27.378813    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:27.417818 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:27.417786    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn"]
Oct 30 22:18:27.418086 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:18:27.418062    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaec98483_f8d7_4eb3_811e_5f6b86e56230.slice/crio-777e50887d2b51f53f1c6e45152ef7caa12ea93353a1c8ba61549e5327270ee3 WatchSource:0}: Error finding container 777e50887d2b51f53f1c6e45152ef7caa12ea93353a1c8ba61549e5327270ee3: Status 404 returned error can't find the container with id 777e50887d2b51f53f1c6e45152ef7caa12ea93353a1c8ba61549e5327270ee3
Oct 30 22:18:27.579863 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:27.579823    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"]
Oct 30 22:18:27.580128 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:27.580100    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w" podUID="73ac22d3-ce8c-4c0c-a742-11d19526d964" containerName="istio-proxy" containerID="cri-o://e786a485f0980baa83e4d1e469cf0a56ff9f458b69756e568ccecd3254e8ad50" gracePeriod=30
Oct 30 22:18:27.639204 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:27.639083    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:27.649560 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:27.649524    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn" event={"ID":"aec98483-f8d7-4eb3-811e-5f6b86e56230","Type":"ContainerStarted","Data":"777e50887d2b51f53f1c6e45152ef7caa12ea93353a1c8ba61549e5327270ee3"}
Oct 30 22:18:28.659768 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:28.659730    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn" event={"ID":"aec98483-f8d7-4eb3-811e-5f6b86e56230","Type":"ContainerStarted","Data":"1e5659f43eee7a0d449a6c665a4f99636ab1cef7504bc6dbe1e592640e45314f"}
Oct 30 22:18:28.660178 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:28.659823    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn"
Oct 30 22:18:28.778149 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:28.778030    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn" podStartSLOduration=1.7507889319999999 podStartE2EDuration="2.778014676s" totalImagesPullingTime="1.027225744s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:28.776094057 +0000 UTC m=+2450.953582674" watchObservedRunningTime="2026-10-30 22:18:28.778014676 +0000 UTC m=+2450.955503328"
Oct 30 22:18:31.449027 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:31.448979    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:32.650744 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:32.650707    4871 patch_prober.go:29] interesting pod/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.142:15021/healthz/ready\": dial tcp 10.133.0.142:15021: connect: connection refused" start-of-body=
Oct 30 22:18:32.651228 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:32.650790    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w" podUID="73ac22d3-ce8c-4c0c-a742-11d19526d964" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.142:15021/healthz/ready\": dial tcp 10.133.0.142:15021: connect: connection refused"
Oct 30 22:18:32.872677 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:32.872650    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"
Oct 30 22:18:32.979813 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:32.979774    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73ac22d3-ce8c-4c0c-a742-11d19526d964-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73ac22d3-ce8c-4c0c-a742-11d19526d964-credential-socket\") pod \"73ac22d3-ce8c-4c0c-a742-11d19526d964\" (UID: \"73ac22d3-ce8c-4c0c-a742-11d19526d964\") "
Oct 30 22:18:32.980008 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:32.979991    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/73ac22d3-ce8c-4c0c-a742-11d19526d964-istio-token\" (UniqueName: \"kubernetes.io/projected/73ac22d3-ce8c-4c0c-a742-11d19526d964-istio-token\") pod \"73ac22d3-ce8c-4c0c-a742-11d19526d964\" (UID: \"73ac22d3-ce8c-4c0c-a742-11d19526d964\") "
Oct 30 22:18:32.980117 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:32.980101    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/73ac22d3-ce8c-4c0c-a742-11d19526d964-kube-api-access-nvgq9\" (UniqueName: \"kubernetes.io/projected/73ac22d3-ce8c-4c0c-a742-11d19526d964-kube-api-access-nvgq9\") pod \"73ac22d3-ce8c-4c0c-a742-11d19526d964\" (UID: \"73ac22d3-ce8c-4c0c-a742-11d19526d964\") "
Oct 30 22:18:32.980159 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:32.980144    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73ac22d3-ce8c-4c0c-a742-11d19526d964-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73ac22d3-ce8c-4c0c-a742-11d19526d964-istio-data\") pod \"73ac22d3-ce8c-4c0c-a742-11d19526d964\" (UID: \"73ac22d3-ce8c-4c0c-a742-11d19526d964\") "
Oct 30 22:18:32.980197 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:32.980179    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73ac22d3-ce8c-4c0c-a742-11d19526d964-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73ac22d3-ce8c-4c0c-a742-11d19526d964-istio-envoy\") pod \"73ac22d3-ce8c-4c0c-a742-11d19526d964\" (UID: \"73ac22d3-ce8c-4c0c-a742-11d19526d964\") "
Oct 30 22:18:32.980520 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:32.980230    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/73ac22d3-ce8c-4c0c-a742-11d19526d964-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73ac22d3-ce8c-4c0c-a742-11d19526d964-istio-podinfo\") pod \"73ac22d3-ce8c-4c0c-a742-11d19526d964\" (UID: \"73ac22d3-ce8c-4c0c-a742-11d19526d964\") "
Oct 30 22:18:32.980520 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:32.980312    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73ac22d3-ce8c-4c0c-a742-11d19526d964-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73ac22d3-ce8c-4c0c-a742-11d19526d964-workload-socket\") pod \"73ac22d3-ce8c-4c0c-a742-11d19526d964\" (UID: \"73ac22d3-ce8c-4c0c-a742-11d19526d964\") "
Oct 30 22:18:32.980520 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:32.980264    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73ac22d3-ce8c-4c0c-a742-11d19526d964-credential-socket" pod "73ac22d3-ce8c-4c0c-a742-11d19526d964" (UID: "73ac22d3-ce8c-4c0c-a742-11d19526d964"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:32.980520 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:32.980345    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73ac22d3-ce8c-4c0c-a742-11d19526d964-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73ac22d3-ce8c-4c0c-a742-11d19526d964-workload-certs\") pod \"73ac22d3-ce8c-4c0c-a742-11d19526d964\" (UID: \"73ac22d3-ce8c-4c0c-a742-11d19526d964\") "
Oct 30 22:18:32.980520 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:32.980509    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73ac22d3-ce8c-4c0c-a742-11d19526d964-istio-data" pod "73ac22d3-ce8c-4c0c-a742-11d19526d964" (UID: "73ac22d3-ce8c-4c0c-a742-11d19526d964"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:32.980913 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:32.980525    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/73ac22d3-ce8c-4c0c-a742-11d19526d964-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73ac22d3-ce8c-4c0c-a742-11d19526d964-istiod-ca-cert\") pod \"73ac22d3-ce8c-4c0c-a742-11d19526d964\" (UID: \"73ac22d3-ce8c-4c0c-a742-11d19526d964\") "
Oct 30 22:18:32.980913 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:32.980680    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73ac22d3-ce8c-4c0c-a742-11d19526d964-workload-socket" pod "73ac22d3-ce8c-4c0c-a742-11d19526d964" (UID: "73ac22d3-ce8c-4c0c-a742-11d19526d964"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:32.981028 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:32.980986    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73ac22d3-ce8c-4c0c-a742-11d19526d964-istiod-ca-cert" pod "73ac22d3-ce8c-4c0c-a742-11d19526d964" (UID: "73ac22d3-ce8c-4c0c-a742-11d19526d964"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:18:32.981092 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:32.981050    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73ac22d3-ce8c-4c0c-a742-11d19526d964-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:32.981092 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:32.981070    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73ac22d3-ce8c-4c0c-a742-11d19526d964-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:32.981092 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:32.981083    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73ac22d3-ce8c-4c0c-a742-11d19526d964-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:32.981277 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:32.981158    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73ac22d3-ce8c-4c0c-a742-11d19526d964-workload-certs" pod "73ac22d3-ce8c-4c0c-a742-11d19526d964" (UID: "73ac22d3-ce8c-4c0c-a742-11d19526d964"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:32.982536 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:32.982514    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/73ac22d3-ce8c-4c0c-a742-11d19526d964-istio-podinfo" pod "73ac22d3-ce8c-4c0c-a742-11d19526d964" (UID: "73ac22d3-ce8c-4c0c-a742-11d19526d964"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:18:32.982622 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:32.982533    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73ac22d3-ce8c-4c0c-a742-11d19526d964-istio-token" pod "73ac22d3-ce8c-4c0c-a742-11d19526d964" (UID: "73ac22d3-ce8c-4c0c-a742-11d19526d964"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:18:32.982814 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:32.982794    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73ac22d3-ce8c-4c0c-a742-11d19526d964-istio-envoy" pod "73ac22d3-ce8c-4c0c-a742-11d19526d964" (UID: "73ac22d3-ce8c-4c0c-a742-11d19526d964"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:32.982869 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:32.982854    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73ac22d3-ce8c-4c0c-a742-11d19526d964-kube-api-access-nvgq9" pod "73ac22d3-ce8c-4c0c-a742-11d19526d964" (UID: "73ac22d3-ce8c-4c0c-a742-11d19526d964"). InnerVolumeSpecName "kube-api-access-nvgq9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:18:33.082099 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:33.082049    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvgq9\" (UniqueName: \"kubernetes.io/projected/73ac22d3-ce8c-4c0c-a742-11d19526d964-kube-api-access-nvgq9\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:33.082099 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:33.082090    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73ac22d3-ce8c-4c0c-a742-11d19526d964-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:33.082099 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:33.082104    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73ac22d3-ce8c-4c0c-a742-11d19526d964-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:33.082406 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:33.082116    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73ac22d3-ce8c-4c0c-a742-11d19526d964-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:33.082406 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:33.082130    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73ac22d3-ce8c-4c0c-a742-11d19526d964-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:33.082406 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:33.082141    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73ac22d3-ce8c-4c0c-a742-11d19526d964-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:33.705618 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:33.705575    4871 generic.go:356] "Generic (PLEG): container finished" podID="73ac22d3-ce8c-4c0c-a742-11d19526d964" containerID="e786a485f0980baa83e4d1e469cf0a56ff9f458b69756e568ccecd3254e8ad50" exitCode=0
Oct 30 22:18:33.706149 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:33.705658    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"
Oct 30 22:18:33.706149 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:33.705904    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w" event={"ID":"73ac22d3-ce8c-4c0c-a742-11d19526d964","Type":"ContainerDied","Data":"e786a485f0980baa83e4d1e469cf0a56ff9f458b69756e568ccecd3254e8ad50"}
Oct 30 22:18:33.706149 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:33.705990    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w" event={"ID":"73ac22d3-ce8c-4c0c-a742-11d19526d964","Type":"ContainerDied","Data":"b22e22a57f8d41b9cdc23c32238cd391f5a3f8f3262f438c71e03378c38fa3b0"}
Oct 30 22:18:33.706149 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:33.706024    4871 scope.go:122] "RemoveContainer" containerID="e786a485f0980baa83e4d1e469cf0a56ff9f458b69756e568ccecd3254e8ad50"
Oct 30 22:18:33.722794 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:33.722765    4871 scope.go:122] "RemoveContainer" containerID="fd2b871d1f0d096fa2de6aa11147aa921d493260eabc795eb659d47b61da8ba5"
Oct 30 22:18:33.737414 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:33.737376    4871 scope.go:122] "RemoveContainer" containerID="e786a485f0980baa83e4d1e469cf0a56ff9f458b69756e568ccecd3254e8ad50"
Oct 30 22:18:33.737760 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:18:33.737732    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e786a485f0980baa83e4d1e469cf0a56ff9f458b69756e568ccecd3254e8ad50\": container with ID starting with e786a485f0980baa83e4d1e469cf0a56ff9f458b69756e568ccecd3254e8ad50 not found: ID does not exist" containerID="e786a485f0980baa83e4d1e469cf0a56ff9f458b69756e568ccecd3254e8ad50"
Oct 30 22:18:33.737881 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:33.737767    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e786a485f0980baa83e4d1e469cf0a56ff9f458b69756e568ccecd3254e8ad50"} err="failed to get container status \"e786a485f0980baa83e4d1e469cf0a56ff9f458b69756e568ccecd3254e8ad50\": rpc error: code = NotFound desc = could not find container \"e786a485f0980baa83e4d1e469cf0a56ff9f458b69756e568ccecd3254e8ad50\": container with ID starting with e786a485f0980baa83e4d1e469cf0a56ff9f458b69756e568ccecd3254e8ad50 not found: ID does not exist"
Oct 30 22:18:33.737881 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:33.737791    4871 scope.go:122] "RemoveContainer" containerID="fd2b871d1f0d096fa2de6aa11147aa921d493260eabc795eb659d47b61da8ba5"
Oct 30 22:18:33.738161 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:18:33.738135    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd2b871d1f0d096fa2de6aa11147aa921d493260eabc795eb659d47b61da8ba5\": container with ID starting with fd2b871d1f0d096fa2de6aa11147aa921d493260eabc795eb659d47b61da8ba5 not found: ID does not exist" containerID="fd2b871d1f0d096fa2de6aa11147aa921d493260eabc795eb659d47b61da8ba5"
Oct 30 22:18:33.738610 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:33.738168    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd2b871d1f0d096fa2de6aa11147aa921d493260eabc795eb659d47b61da8ba5"} err="failed to get container status \"fd2b871d1f0d096fa2de6aa11147aa921d493260eabc795eb659d47b61da8ba5\": rpc error: code = NotFound desc = could not find container \"fd2b871d1f0d096fa2de6aa11147aa921d493260eabc795eb659d47b61da8ba5\": container with ID starting with fd2b871d1f0d096fa2de6aa11147aa921d493260eabc795eb659d47b61da8ba5 not found: ID does not exist"
Oct 30 22:18:33.914748 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:33.914705    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"]
Oct 30 22:18:33.966317 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:33.966223    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-95b84c-openshift-default-68fd98686f-phw5w"]
Oct 30 22:18:34.674029 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:34.674000    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn"
Oct 30 22:18:35.568894 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:18:35.568857    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:18:35.962494 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:35.962450    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="73ac22d3-ce8c-4c0c-a742-11d19526d964" path="/var/lib/kubelet/pods/73ac22d3-ce8c-4c0c-a742-11d19526d964/volumes"
Oct 30 22:18:36.301115 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:36.301023    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:37.005192 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:37.005158    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:37.160185 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:37.160135    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:40.255623 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:40.255584    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj"]
Oct 30 22:18:40.256102 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:40.255791    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj" podUID="3df5f9f5-7977-43db-be67-e1105d82e626" containerName="maas-api" containerID="cri-o://82763cfaca8b7c0bd3fe093bade43f098bbad998c56ae4b621c6d176798554a8" gracePeriod=30
Oct 30 22:18:40.359917 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:40.359874    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:40.568054 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:40.568028    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj"
Oct 30 22:18:40.688262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:40.687619    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3df5f9f5-7977-43db-be67-e1105d82e626-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3df5f9f5-7977-43db-be67-e1105d82e626-openshift-service-ca\") pod \"3df5f9f5-7977-43db-be67-e1105d82e626\" (UID: \"3df5f9f5-7977-43db-be67-e1105d82e626\") "
Oct 30 22:18:40.688262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:40.687814    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3df5f9f5-7977-43db-be67-e1105d82e626-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3df5f9f5-7977-43db-be67-e1105d82e626-metrics-certs\") pod \"3df5f9f5-7977-43db-be67-e1105d82e626\" (UID: \"3df5f9f5-7977-43db-be67-e1105d82e626\") "
Oct 30 22:18:40.688262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:40.687899    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3df5f9f5-7977-43db-be67-e1105d82e626-kube-api-access-4669d\" (UniqueName: \"kubernetes.io/projected/3df5f9f5-7977-43db-be67-e1105d82e626-kube-api-access-4669d\") pod \"3df5f9f5-7977-43db-be67-e1105d82e626\" (UID: \"3df5f9f5-7977-43db-be67-e1105d82e626\") "
Oct 30 22:18:40.688262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:40.688009    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3df5f9f5-7977-43db-be67-e1105d82e626-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3df5f9f5-7977-43db-be67-e1105d82e626-maas-api-tls\") pod \"3df5f9f5-7977-43db-be67-e1105d82e626\" (UID: \"3df5f9f5-7977-43db-be67-e1105d82e626\") "
Oct 30 22:18:40.689733 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:40.689699    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3df5f9f5-7977-43db-be67-e1105d82e626-openshift-service-ca" pod "3df5f9f5-7977-43db-be67-e1105d82e626" (UID: "3df5f9f5-7977-43db-be67-e1105d82e626"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:18:40.705854 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:40.705817    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3df5f9f5-7977-43db-be67-e1105d82e626-maas-api-tls" pod "3df5f9f5-7977-43db-be67-e1105d82e626" (UID: "3df5f9f5-7977-43db-be67-e1105d82e626"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:18:40.706805 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:40.706381    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3df5f9f5-7977-43db-be67-e1105d82e626-kube-api-access-4669d" pod "3df5f9f5-7977-43db-be67-e1105d82e626" (UID: "3df5f9f5-7977-43db-be67-e1105d82e626"). InnerVolumeSpecName "kube-api-access-4669d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:18:40.707366 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:40.707337    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3df5f9f5-7977-43db-be67-e1105d82e626-metrics-certs" pod "3df5f9f5-7977-43db-be67-e1105d82e626" (UID: "3df5f9f5-7977-43db-be67-e1105d82e626"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:18:40.767855 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:40.767813    4871 generic.go:356] "Generic (PLEG): container finished" podID="3df5f9f5-7977-43db-be67-e1105d82e626" containerID="82763cfaca8b7c0bd3fe093bade43f098bbad998c56ae4b621c6d176798554a8" exitCode=0
Oct 30 22:18:40.768033 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:40.767869    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj"
Oct 30 22:18:40.768033 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:40.767894    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj" event={"ID":"3df5f9f5-7977-43db-be67-e1105d82e626","Type":"ContainerDied","Data":"82763cfaca8b7c0bd3fe093bade43f098bbad998c56ae4b621c6d176798554a8"}
Oct 30 22:18:40.768033 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:40.767975    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj" event={"ID":"3df5f9f5-7977-43db-be67-e1105d82e626","Type":"ContainerDied","Data":"7f2ff8b7e1ed7d6f654b8369bdaf573b25444a462df301d33ed3617ec2e4dbaa"}
Oct 30 22:18:40.768033 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:40.768003    4871 scope.go:122] "RemoveContainer" containerID="82763cfaca8b7c0bd3fe093bade43f098bbad998c56ae4b621c6d176798554a8"
Oct 30 22:18:40.780885 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:40.780856    4871 scope.go:122] "RemoveContainer" containerID="82763cfaca8b7c0bd3fe093bade43f098bbad998c56ae4b621c6d176798554a8"
Oct 30 22:18:40.781215 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:18:40.781172    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82763cfaca8b7c0bd3fe093bade43f098bbad998c56ae4b621c6d176798554a8\": container with ID starting with 82763cfaca8b7c0bd3fe093bade43f098bbad998c56ae4b621c6d176798554a8 not found: ID does not exist" containerID="82763cfaca8b7c0bd3fe093bade43f098bbad998c56ae4b621c6d176798554a8"
Oct 30 22:18:40.781215 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:40.781199    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82763cfaca8b7c0bd3fe093bade43f098bbad998c56ae4b621c6d176798554a8"} err="failed to get container status \"82763cfaca8b7c0bd3fe093bade43f098bbad998c56ae4b621c6d176798554a8\": rpc error: code = NotFound desc = could not find container \"82763cfaca8b7c0bd3fe093bade43f098bbad998c56ae4b621c6d176798554a8\": container with ID starting with 82763cfaca8b7c0bd3fe093bade43f098bbad998c56ae4b621c6d176798554a8 not found: ID does not exist"
Oct 30 22:18:40.789312 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:40.789283    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4669d\" (UniqueName: \"kubernetes.io/projected/3df5f9f5-7977-43db-be67-e1105d82e626-kube-api-access-4669d\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:40.789312 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:40.789314    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3df5f9f5-7977-43db-be67-e1105d82e626-maas-api-tls\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:40.789486 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:40.789325    4871 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3df5f9f5-7977-43db-be67-e1105d82e626-openshift-service-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:40.789486 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:40.789335    4871 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3df5f9f5-7977-43db-be67-e1105d82e626-metrics-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:40.872033 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:40.871993    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:40.928368 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:40.928334    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj"]
Oct 30 22:18:40.934196 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:40.934163    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f0b8c9-57d968779b-tqdcj"]
Oct 30 22:18:41.029474 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:41.029431    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:41.519041 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:41.519003    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:41.899739 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:41.899694    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:41.922831 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:41.922794    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-138566-6ffd477879-wbs5f"]
Oct 30 22:18:41.923552 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:41.923515    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-138566-6ffd477879-wbs5f" podUID="5937031a-6886-437b-98e2-6db098950b8c" containerName="payload-pre-processing" containerID="cri-o://8d66dc380b46e446b35b11ec3b92d3fd98c4022cf3cb012dcf255a1c8f226866" gracePeriod=30
Oct 30 22:18:41.961267 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:41.961218    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3df5f9f5-7977-43db-be67-e1105d82e626" path="/var/lib/kubelet/pods/3df5f9f5-7977-43db-be67-e1105d82e626/volumes"
Oct 30 22:18:42.063006 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:42.062951    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:42.203617 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:42.203555    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-138566-6ffd477879-wbs5f"
Oct 30 22:18:42.214855 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:42.214822    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:42.254682 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:42.254647    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5937031a-6886-437b-98e2-6db098950b8c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5937031a-6886-437b-98e2-6db098950b8c-plugins-config-volume\") pod \"5937031a-6886-437b-98e2-6db098950b8c\" (UID: \"5937031a-6886-437b-98e2-6db098950b8c\") "
Oct 30 22:18:42.254872 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:42.254745    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5937031a-6886-437b-98e2-6db098950b8c-kube-api-access-q9vnp\" (UniqueName: \"kubernetes.io/projected/5937031a-6886-437b-98e2-6db098950b8c-kube-api-access-q9vnp\") pod \"5937031a-6886-437b-98e2-6db098950b8c\" (UID: \"5937031a-6886-437b-98e2-6db098950b8c\") "
Oct 30 22:18:42.255069 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:42.255045    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5937031a-6886-437b-98e2-6db098950b8c-plugins-config-volume" pod "5937031a-6886-437b-98e2-6db098950b8c" (UID: "5937031a-6886-437b-98e2-6db098950b8c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:18:42.255214 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:42.255196    4871 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5937031a-6886-437b-98e2-6db098950b8c-plugins-config-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:42.256915 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:42.256890    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5937031a-6886-437b-98e2-6db098950b8c-kube-api-access-q9vnp" pod "5937031a-6886-437b-98e2-6db098950b8c" (UID: "5937031a-6886-437b-98e2-6db098950b8c"). InnerVolumeSpecName "kube-api-access-q9vnp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:18:42.357260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:42.356428    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9vnp\" (UniqueName: \"kubernetes.io/projected/5937031a-6886-437b-98e2-6db098950b8c-kube-api-access-q9vnp\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:42.671994 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:42.671956    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:42.786447 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:42.786402    4871 generic.go:356] "Generic (PLEG): container finished" podID="5937031a-6886-437b-98e2-6db098950b8c" containerID="8d66dc380b46e446b35b11ec3b92d3fd98c4022cf3cb012dcf255a1c8f226866" exitCode=0
Oct 30 22:18:42.786671 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:42.786467    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-138566-6ffd477879-wbs5f"
Oct 30 22:18:42.786671 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:42.786481    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-138566-6ffd477879-wbs5f" event={"ID":"5937031a-6886-437b-98e2-6db098950b8c","Type":"ContainerDied","Data":"8d66dc380b46e446b35b11ec3b92d3fd98c4022cf3cb012dcf255a1c8f226866"}
Oct 30 22:18:42.786671 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:42.786551    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-138566-6ffd477879-wbs5f" event={"ID":"5937031a-6886-437b-98e2-6db098950b8c","Type":"ContainerDied","Data":"d371f89bb94959f817f33486b7ce7c1200c4b5d19cf988f701a046080ceace8d"}
Oct 30 22:18:42.786671 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:42.786587    4871 scope.go:122] "RemoveContainer" containerID="8d66dc380b46e446b35b11ec3b92d3fd98c4022cf3cb012dcf255a1c8f226866"
Oct 30 22:18:42.797808 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:42.797776    4871 scope.go:122] "RemoveContainer" containerID="8d66dc380b46e446b35b11ec3b92d3fd98c4022cf3cb012dcf255a1c8f226866"
Oct 30 22:18:42.798166 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:18:42.798145    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d66dc380b46e446b35b11ec3b92d3fd98c4022cf3cb012dcf255a1c8f226866\": container with ID starting with 8d66dc380b46e446b35b11ec3b92d3fd98c4022cf3cb012dcf255a1c8f226866 not found: ID does not exist" containerID="8d66dc380b46e446b35b11ec3b92d3fd98c4022cf3cb012dcf255a1c8f226866"
Oct 30 22:18:42.798219 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:42.798175    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d66dc380b46e446b35b11ec3b92d3fd98c4022cf3cb012dcf255a1c8f226866"} err="failed to get container status \"8d66dc380b46e446b35b11ec3b92d3fd98c4022cf3cb012dcf255a1c8f226866\": rpc error: code = NotFound desc = could not find container \"8d66dc380b46e446b35b11ec3b92d3fd98c4022cf3cb012dcf255a1c8f226866\": container with ID starting with 8d66dc380b46e446b35b11ec3b92d3fd98c4022cf3cb012dcf255a1c8f226866 not found: ID does not exist"
Oct 30 22:18:42.858253 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:42.858207    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:42.943972 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:42.943876    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-138566-6ffd477879-wbs5f"]
Oct 30 22:18:42.947933 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:42.947902    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-138566-6ffd477879-wbs5f"]
Oct 30 22:18:43.087679 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:43.087642    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:43.342420 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:43.342321    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:43.502392 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:43.502032    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:43.961540 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:43.961506    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5937031a-6886-437b-98e2-6db098950b8c" path="/var/lib/kubelet/pods/5937031a-6886-437b-98e2-6db098950b8c/volumes"
Oct 30 22:18:44.099465 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:44.099431    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:48.306929 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:48.306845    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:48.627192 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:48.627155    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:48.795013 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:48.794970    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:48.971555 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:48.971458    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"]
Oct 30 22:18:48.971719 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:48.971676    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj" podUID="22d76305-7ecb-421f-9926-b1de7d91d2d8" containerName="istio-proxy" containerID="cri-o://7375cf92a0a97662af605be144db34dda8698b0ef9952014400b80cd21e7c659" gracePeriod=30
Oct 30 22:18:48.978014 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:48.977985    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:50.542340 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:50.542299    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:50.710216 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:50.707327    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:50.758899 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:50.758869    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"]
Oct 30 22:18:50.759095 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:50.759075    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4" podUID="4a9121ce-2d96-4f01-8463-ef1768b5f588" containerName="istio-proxy" containerID="cri-o://600a071749ac5e0db389e71f1fc3831ab4e7ca9421ad5a984cf10b06d9f56705" gracePeriod=30
Oct 30 22:18:50.885975 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:50.881953    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:52.110775 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:52.110740    4871 patch_prober.go:29] interesting pod/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:18:52.111179 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:52.110811    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj" podUID="22d76305-7ecb-421f-9926-b1de7d91d2d8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:18:52.385652 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:52.385603    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:52.657215 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:52.657118    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df"]
Oct 30 22:18:52.657449 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:52.657402    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df" podUID="41ad0caa-9e1c-4dc1-9f70-43beb36de4a7" containerName="maas-api" containerID="cri-o://7024a37a67b8de382f51d56ec9a5df0dd3f2b762d6c0a2a67c8c49bd1729c9e0" gracePeriod=30
Oct 30 22:18:52.767197 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:52.767162    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:52.899456 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:52.899421    4871 generic.go:356] "Generic (PLEG): container finished" podID="41ad0caa-9e1c-4dc1-9f70-43beb36de4a7" containerID="7024a37a67b8de382f51d56ec9a5df0dd3f2b762d6c0a2a67c8c49bd1729c9e0" exitCode=0
Oct 30 22:18:52.899659 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:52.899501    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df" event={"ID":"41ad0caa-9e1c-4dc1-9f70-43beb36de4a7","Type":"ContainerDied","Data":"7024a37a67b8de382f51d56ec9a5df0dd3f2b762d6c0a2a67c8c49bd1729c9e0"}
Oct 30 22:18:52.925129 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:52.925095    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"]
Oct 30 22:18:52.926036 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:52.926014    4871 cpu_manager.go:414] "containerMap: removing container" podUID="3df5f9f5-7977-43db-be67-e1105d82e626" containerName="maas-api"
Oct 30 22:18:52.926106 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:52.926040    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3df5f9f5-7977-43db-be67-e1105d82e626" containerName="maas-api"
Oct 30 22:18:52.926106 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:52.926066    4871 cpu_manager.go:414] "containerMap: removing container" podUID="5937031a-6886-437b-98e2-6db098950b8c" containerName="payload-pre-processing"
Oct 30 22:18:52.926106 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:52.926074    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5937031a-6886-437b-98e2-6db098950b8c" containerName="payload-pre-processing"
Oct 30 22:18:52.926204 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:52.926141    4871 cpu_manager.go:414] "containerMap: removing container" podUID="f9b60b37-4047-499d-87fa-2533d0e12e85" containerName="istio-proxy"
Oct 30 22:18:52.926204 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:52.926149    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f9b60b37-4047-499d-87fa-2533d0e12e85" containerName="istio-proxy"
Oct 30 22:18:52.926204 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:52.926163    4871 cpu_manager.go:414] "containerMap: removing container" podUID="73ac22d3-ce8c-4c0c-a742-11d19526d964" containerName="istio-proxy"
Oct 30 22:18:52.926204 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:52.926170    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="73ac22d3-ce8c-4c0c-a742-11d19526d964" containerName="istio-proxy"
Oct 30 22:18:52.926204 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:52.926185    4871 cpu_manager.go:414] "containerMap: removing container" podUID="73ac22d3-ce8c-4c0c-a742-11d19526d964" containerName="istio-proxy"
Oct 30 22:18:52.926204 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:52.926193    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="73ac22d3-ce8c-4c0c-a742-11d19526d964" containerName="istio-proxy"
Oct 30 22:18:52.926436 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:52.926334    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="f9b60b37-4047-499d-87fa-2533d0e12e85" containerName="istio-proxy"
Oct 30 22:18:52.926436 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:52.926349    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="73ac22d3-ce8c-4c0c-a742-11d19526d964" containerName="istio-proxy"
Oct 30 22:18:52.926436 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:52.926362    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="73ac22d3-ce8c-4c0c-a742-11d19526d964" containerName="istio-proxy"
Oct 30 22:18:52.926436 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:52.926373    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="3df5f9f5-7977-43db-be67-e1105d82e626" containerName="maas-api"
Oct 30 22:18:52.926436 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:52.926385    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="5937031a-6886-437b-98e2-6db098950b8c" containerName="payload-pre-processing"
Oct 30 22:18:52.930687 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:52.930665    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"
Oct 30 22:18:52.936811 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:52.936790    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df"
Oct 30 22:18:52.989283 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:52.983025    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-a-3cecc7-openshift-default-dockercfg-9bxgs\""
Oct 30 22:18:53.011444 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:52.998487    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/41ad0caa-9e1c-4dc1-9f70-43beb36de4a7-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41ad0caa-9e1c-4dc1-9f70-43beb36de4a7-maas-api-tls\") pod \"41ad0caa-9e1c-4dc1-9f70-43beb36de4a7\" (UID: \"41ad0caa-9e1c-4dc1-9f70-43beb36de4a7\") "
Oct 30 22:18:53.011444 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:52.998549    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/41ad0caa-9e1c-4dc1-9f70-43beb36de4a7-metrics-certs\" (UniqueName: \"kubernetes.io/secret/41ad0caa-9e1c-4dc1-9f70-43beb36de4a7-metrics-certs\") pod \"41ad0caa-9e1c-4dc1-9f70-43beb36de4a7\" (UID: \"41ad0caa-9e1c-4dc1-9f70-43beb36de4a7\") "
Oct 30 22:18:53.011444 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:52.998679    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/41ad0caa-9e1c-4dc1-9f70-43beb36de4a7-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41ad0caa-9e1c-4dc1-9f70-43beb36de4a7-openshift-service-ca\") pod \"41ad0caa-9e1c-4dc1-9f70-43beb36de4a7\" (UID: \"41ad0caa-9e1c-4dc1-9f70-43beb36de4a7\") "
Oct 30 22:18:53.011444 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:52.998720    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/41ad0caa-9e1c-4dc1-9f70-43beb36de4a7-kube-api-access-vzhjc\" (UniqueName: \"kubernetes.io/projected/41ad0caa-9e1c-4dc1-9f70-43beb36de4a7-kube-api-access-vzhjc\") pod \"41ad0caa-9e1c-4dc1-9f70-43beb36de4a7\" (UID: \"41ad0caa-9e1c-4dc1-9f70-43beb36de4a7\") "
Oct 30 22:18:53.011444 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:52.999142    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fad116b-e1cc-42da-a439-bdecb46b96a9-istio-data\") pod \"e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb\" (UID: \"1fad116b-e1cc-42da-a439-bdecb46b96a9\") " pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"
Oct 30 22:18:53.011444 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:52.999186    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fad116b-e1cc-42da-a439-bdecb46b96a9-credential-socket\") pod \"e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb\" (UID: \"1fad116b-e1cc-42da-a439-bdecb46b96a9\") " pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"
Oct 30 22:18:53.011444 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:52.999454    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fad116b-e1cc-42da-a439-bdecb46b96a9-istio-token\") pod \"e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb\" (UID: \"1fad116b-e1cc-42da-a439-bdecb46b96a9\") " pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"
Oct 30 22:18:53.011444 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:52.999501    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fad116b-e1cc-42da-a439-bdecb46b96a9-istio-podinfo\") pod \"e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb\" (UID: \"1fad116b-e1cc-42da-a439-bdecb46b96a9\") " pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"
Oct 30 22:18:53.011444 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:52.999547    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fad116b-e1cc-42da-a439-bdecb46b96a9-workload-certs\") pod \"e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb\" (UID: \"1fad116b-e1cc-42da-a439-bdecb46b96a9\") " pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"
Oct 30 22:18:53.011444 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:52.999575    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fad116b-e1cc-42da-a439-bdecb46b96a9-istiod-ca-cert\") pod \"e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb\" (UID: \"1fad116b-e1cc-42da-a439-bdecb46b96a9\") " pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"
Oct 30 22:18:53.011444 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:52.999648    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fad116b-e1cc-42da-a439-bdecb46b96a9-workload-socket\") pod \"e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb\" (UID: \"1fad116b-e1cc-42da-a439-bdecb46b96a9\") " pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"
Oct 30 22:18:53.011444 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:52.999686    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sdhr\" (UniqueName: \"kubernetes.io/projected/1fad116b-e1cc-42da-a439-bdecb46b96a9-kube-api-access-5sdhr\") pod \"e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb\" (UID: \"1fad116b-e1cc-42da-a439-bdecb46b96a9\") " pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"
Oct 30 22:18:53.011444 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:52.999794    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fad116b-e1cc-42da-a439-bdecb46b96a9-istio-envoy\") pod \"e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb\" (UID: \"1fad116b-e1cc-42da-a439-bdecb46b96a9\") " pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"
Oct 30 22:18:53.011444 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:53.005915    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41ad0caa-9e1c-4dc1-9f70-43beb36de4a7-openshift-service-ca" pod "41ad0caa-9e1c-4dc1-9f70-43beb36de4a7" (UID: "41ad0caa-9e1c-4dc1-9f70-43beb36de4a7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:18:53.030278 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:53.027744    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:53.067856 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:53.067526    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"]
Oct 30 22:18:53.073505 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:53.073415    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41ad0caa-9e1c-4dc1-9f70-43beb36de4a7-maas-api-tls" pod "41ad0caa-9e1c-4dc1-9f70-43beb36de4a7" (UID: "41ad0caa-9e1c-4dc1-9f70-43beb36de4a7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:18:53.077134 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:53.077109    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41ad0caa-9e1c-4dc1-9f70-43beb36de4a7-metrics-certs" pod "41ad0caa-9e1c-4dc1-9f70-43beb36de4a7" (UID: "41ad0caa-9e1c-4dc1-9f70-43beb36de4a7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:18:53.077227 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:53.077108    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41ad0caa-9e1c-4dc1-9f70-43beb36de4a7-kube-api-access-vzhjc" pod "41ad0caa-9e1c-4dc1-9f70-43beb36de4a7" (UID: "41ad0caa-9e1c-4dc1-9f70-43beb36de4a7"). InnerVolumeSpecName "kube-api-access-vzhjc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:18:53.100973 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:53.100929    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fad116b-e1cc-42da-a439-bdecb46b96a9-workload-socket\") pod \"e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb\" (UID: \"1fad116b-e1cc-42da-a439-bdecb46b96a9\") " pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"
Oct 30 22:18:53.100973 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:53.100977    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sdhr\" (UniqueName: \"kubernetes.io/projected/1fad116b-e1cc-42da-a439-bdecb46b96a9-kube-api-access-5sdhr\") pod \"e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb\" (UID: \"1fad116b-e1cc-42da-a439-bdecb46b96a9\") " pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"
Oct 30 22:18:53.101288 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:53.101028    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fad116b-e1cc-42da-a439-bdecb46b96a9-istio-envoy\") pod \"e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb\" (UID: \"1fad116b-e1cc-42da-a439-bdecb46b96a9\") " pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"
Oct 30 22:18:53.101288 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:53.101087    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fad116b-e1cc-42da-a439-bdecb46b96a9-istio-data\") pod \"e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb\" (UID: \"1fad116b-e1cc-42da-a439-bdecb46b96a9\") " pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"
Oct 30 22:18:53.101288 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:53.101108    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fad116b-e1cc-42da-a439-bdecb46b96a9-credential-socket\") pod \"e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb\" (UID: \"1fad116b-e1cc-42da-a439-bdecb46b96a9\") " pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"
Oct 30 22:18:53.101288 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:53.101168    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fad116b-e1cc-42da-a439-bdecb46b96a9-istio-token\") pod \"e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb\" (UID: \"1fad116b-e1cc-42da-a439-bdecb46b96a9\") " pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"
Oct 30 22:18:53.101288 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:53.101194    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fad116b-e1cc-42da-a439-bdecb46b96a9-istio-podinfo\") pod \"e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb\" (UID: \"1fad116b-e1cc-42da-a439-bdecb46b96a9\") " pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"
Oct 30 22:18:53.101288 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:53.101219    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fad116b-e1cc-42da-a439-bdecb46b96a9-workload-certs\") pod \"e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb\" (UID: \"1fad116b-e1cc-42da-a439-bdecb46b96a9\") " pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"
Oct 30 22:18:53.101288 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:53.101275    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fad116b-e1cc-42da-a439-bdecb46b96a9-istiod-ca-cert\") pod \"e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb\" (UID: \"1fad116b-e1cc-42da-a439-bdecb46b96a9\") " pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"
Oct 30 22:18:53.101690 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:53.101355    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41ad0caa-9e1c-4dc1-9f70-43beb36de4a7-maas-api-tls\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:53.101690 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:53.101373    4871 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41ad0caa-9e1c-4dc1-9f70-43beb36de4a7-metrics-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:53.101690 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:53.101389    4871 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41ad0caa-9e1c-4dc1-9f70-43beb36de4a7-openshift-service-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:53.101690 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:53.101403    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzhjc\" (UniqueName: \"kubernetes.io/projected/41ad0caa-9e1c-4dc1-9f70-43beb36de4a7-kube-api-access-vzhjc\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:53.102309 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:53.102231    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fad116b-e1cc-42da-a439-bdecb46b96a9-workload-certs\") pod \"e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb\" (UID: \"1fad116b-e1cc-42da-a439-bdecb46b96a9\") " pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"
Oct 30 22:18:53.102427 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:53.102276    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fad116b-e1cc-42da-a439-bdecb46b96a9-istiod-ca-cert\") pod \"e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb\" (UID: \"1fad116b-e1cc-42da-a439-bdecb46b96a9\") " pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"
Oct 30 22:18:53.102486 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:53.102445    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fad116b-e1cc-42da-a439-bdecb46b96a9-credential-socket\") pod \"e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb\" (UID: \"1fad116b-e1cc-42da-a439-bdecb46b96a9\") " pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"
Oct 30 22:18:53.102600 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:53.102567    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fad116b-e1cc-42da-a439-bdecb46b96a9-istio-data\") pod \"e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb\" (UID: \"1fad116b-e1cc-42da-a439-bdecb46b96a9\") " pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"
Oct 30 22:18:53.103271 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:53.102656    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fad116b-e1cc-42da-a439-bdecb46b96a9-workload-socket\") pod \"e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb\" (UID: \"1fad116b-e1cc-42da-a439-bdecb46b96a9\") " pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"
Oct 30 22:18:53.104614 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:53.104591    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fad116b-e1cc-42da-a439-bdecb46b96a9-istio-podinfo\") pod \"e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb\" (UID: \"1fad116b-e1cc-42da-a439-bdecb46b96a9\") " pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"
Oct 30 22:18:53.104720 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:53.104642    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fad116b-e1cc-42da-a439-bdecb46b96a9-istio-envoy\") pod \"e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb\" (UID: \"1fad116b-e1cc-42da-a439-bdecb46b96a9\") " pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"
Oct 30 22:18:53.159716 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:53.159678    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fad116b-e1cc-42da-a439-bdecb46b96a9-istio-token\") pod \"e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb\" (UID: \"1fad116b-e1cc-42da-a439-bdecb46b96a9\") " pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"
Oct 30 22:18:53.160081 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:53.159787    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sdhr\" (UniqueName: \"kubernetes.io/projected/1fad116b-e1cc-42da-a439-bdecb46b96a9-kube-api-access-5sdhr\") pod \"e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb\" (UID: \"1fad116b-e1cc-42da-a439-bdecb46b96a9\") " pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"
Oct 30 22:18:53.258026 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:53.257810    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"
Oct 30 22:18:53.318988 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:53.314996    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:53.487383 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:53.487334    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:53.785563 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:53.785502    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:18:53.785709 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:53.785608    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:18:53.785709 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:53.785653    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:18:53.797508 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:53.795506    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"]
Oct 30 22:18:53.938354 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:53.938300    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df"
Oct 30 22:18:53.939275 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:53.938589    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df" event={"ID":"41ad0caa-9e1c-4dc1-9f70-43beb36de4a7","Type":"ContainerDied","Data":"a0c2837107c4dd6c0e3b9b3068b2dbc13471fd6a2b0ccf82bc758695c355a32e"}
Oct 30 22:18:53.939493 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:53.939477    4871 scope.go:122] "RemoveContainer" containerID="7024a37a67b8de382f51d56ec9a5df0dd3f2b762d6c0a2a67c8c49bd1729c9e0"
Oct 30 22:18:53.943414 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:53.943382    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb" event={"ID":"1fad116b-e1cc-42da-a439-bdecb46b96a9","Type":"ContainerStarted","Data":"1e5b5fb524e298a03474138035bc46c19a76048153e7436c74e31980f271f959"}
Oct 30 22:18:54.108337 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.108284    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df"]
Oct 30 22:18:54.155008 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.154969    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-65659d-65cf4f6669-cr9df"]
Oct 30 22:18:54.224263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.224126    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb" podStartSLOduration=2.224105057 podStartE2EDuration="2.224105057s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:18:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:54.22038175 +0000 UTC m=+2476.397870367" watchObservedRunningTime="2026-10-30 22:18:54.224105057 +0000 UTC m=+2476.401593669"
Oct 30 22:18:54.258702 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.258671    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"
Oct 30 22:18:54.260375 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.260341    4871 patch_prober.go:29] interesting pod/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.158:15021/healthz/ready\": dial tcp 10.133.0.158:15021: connect: connection refused" start-of-body=
Oct 30 22:18:54.260489 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.260408    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb" podUID="1fad116b-e1cc-42da-a439-bdecb46b96a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.158:15021/healthz/ready\": dial tcp 10.133.0.158:15021: connect: connection refused"
Oct 30 22:18:54.276746 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.276716    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"
Oct 30 22:18:54.378215 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.378170    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/22d76305-7ecb-421f-9926-b1de7d91d2d8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22d76305-7ecb-421f-9926-b1de7d91d2d8-workload-certs\") pod \"22d76305-7ecb-421f-9926-b1de7d91d2d8\" (UID: \"22d76305-7ecb-421f-9926-b1de7d91d2d8\") "
Oct 30 22:18:54.378508 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.378255    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/22d76305-7ecb-421f-9926-b1de7d91d2d8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22d76305-7ecb-421f-9926-b1de7d91d2d8-istio-envoy\") pod \"22d76305-7ecb-421f-9926-b1de7d91d2d8\" (UID: \"22d76305-7ecb-421f-9926-b1de7d91d2d8\") "
Oct 30 22:18:54.378508 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.378315    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/22d76305-7ecb-421f-9926-b1de7d91d2d8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22d76305-7ecb-421f-9926-b1de7d91d2d8-istiod-ca-cert\") pod \"22d76305-7ecb-421f-9926-b1de7d91d2d8\" (UID: \"22d76305-7ecb-421f-9926-b1de7d91d2d8\") "
Oct 30 22:18:54.378508 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.378429    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/22d76305-7ecb-421f-9926-b1de7d91d2d8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22d76305-7ecb-421f-9926-b1de7d91d2d8-istio-data\") pod \"22d76305-7ecb-421f-9926-b1de7d91d2d8\" (UID: \"22d76305-7ecb-421f-9926-b1de7d91d2d8\") "
Oct 30 22:18:54.378508 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.378494    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/22d76305-7ecb-421f-9926-b1de7d91d2d8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22d76305-7ecb-421f-9926-b1de7d91d2d8-credential-socket\") pod \"22d76305-7ecb-421f-9926-b1de7d91d2d8\" (UID: \"22d76305-7ecb-421f-9926-b1de7d91d2d8\") "
Oct 30 22:18:54.378782 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.378530    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/22d76305-7ecb-421f-9926-b1de7d91d2d8-istio-token\" (UniqueName: \"kubernetes.io/projected/22d76305-7ecb-421f-9926-b1de7d91d2d8-istio-token\") pod \"22d76305-7ecb-421f-9926-b1de7d91d2d8\" (UID: \"22d76305-7ecb-421f-9926-b1de7d91d2d8\") "
Oct 30 22:18:54.378782 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.378576    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22d76305-7ecb-421f-9926-b1de7d91d2d8-workload-certs" pod "22d76305-7ecb-421f-9926-b1de7d91d2d8" (UID: "22d76305-7ecb-421f-9926-b1de7d91d2d8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:54.378782 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.378600    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/22d76305-7ecb-421f-9926-b1de7d91d2d8-kube-api-access-nk7dm\" (UniqueName: \"kubernetes.io/projected/22d76305-7ecb-421f-9926-b1de7d91d2d8-kube-api-access-nk7dm\") pod \"22d76305-7ecb-421f-9926-b1de7d91d2d8\" (UID: \"22d76305-7ecb-421f-9926-b1de7d91d2d8\") "
Oct 30 22:18:54.378782 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.378647    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/22d76305-7ecb-421f-9926-b1de7d91d2d8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22d76305-7ecb-421f-9926-b1de7d91d2d8-workload-socket\") pod \"22d76305-7ecb-421f-9926-b1de7d91d2d8\" (UID: \"22d76305-7ecb-421f-9926-b1de7d91d2d8\") "
Oct 30 22:18:54.378782 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.378713    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/22d76305-7ecb-421f-9926-b1de7d91d2d8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22d76305-7ecb-421f-9926-b1de7d91d2d8-istio-podinfo\") pod \"22d76305-7ecb-421f-9926-b1de7d91d2d8\" (UID: \"22d76305-7ecb-421f-9926-b1de7d91d2d8\") "
Oct 30 22:18:54.378782 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.378755    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22d76305-7ecb-421f-9926-b1de7d91d2d8-istiod-ca-cert" pod "22d76305-7ecb-421f-9926-b1de7d91d2d8" (UID: "22d76305-7ecb-421f-9926-b1de7d91d2d8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:18:54.378782 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.378779    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22d76305-7ecb-421f-9926-b1de7d91d2d8-credential-socket" pod "22d76305-7ecb-421f-9926-b1de7d91d2d8" (UID: "22d76305-7ecb-421f-9926-b1de7d91d2d8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:54.379187 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.378933    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22d76305-7ecb-421f-9926-b1de7d91d2d8-istio-data" pod "22d76305-7ecb-421f-9926-b1de7d91d2d8" (UID: "22d76305-7ecb-421f-9926-b1de7d91d2d8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:54.379187 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.379174    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22d76305-7ecb-421f-9926-b1de7d91d2d8-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:54.379330 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.379192    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22d76305-7ecb-421f-9926-b1de7d91d2d8-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:54.379330 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.379204    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22d76305-7ecb-421f-9926-b1de7d91d2d8-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:54.379330 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.379219    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22d76305-7ecb-421f-9926-b1de7d91d2d8-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:54.379514 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.379410    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22d76305-7ecb-421f-9926-b1de7d91d2d8-workload-socket" pod "22d76305-7ecb-421f-9926-b1de7d91d2d8" (UID: "22d76305-7ecb-421f-9926-b1de7d91d2d8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:54.381285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.381255    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22d76305-7ecb-421f-9926-b1de7d91d2d8-istio-envoy" pod "22d76305-7ecb-421f-9926-b1de7d91d2d8" (UID: "22d76305-7ecb-421f-9926-b1de7d91d2d8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:54.381516 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.381495    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/22d76305-7ecb-421f-9926-b1de7d91d2d8-istio-podinfo" pod "22d76305-7ecb-421f-9926-b1de7d91d2d8" (UID: "22d76305-7ecb-421f-9926-b1de7d91d2d8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:18:54.381730 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.381711    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22d76305-7ecb-421f-9926-b1de7d91d2d8-kube-api-access-nk7dm" pod "22d76305-7ecb-421f-9926-b1de7d91d2d8" (UID: "22d76305-7ecb-421f-9926-b1de7d91d2d8"). InnerVolumeSpecName "kube-api-access-nk7dm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:18:54.381924 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.381899    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22d76305-7ecb-421f-9926-b1de7d91d2d8-istio-token" pod "22d76305-7ecb-421f-9926-b1de7d91d2d8" (UID: "22d76305-7ecb-421f-9926-b1de7d91d2d8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:18:54.412830 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.412785    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:54.479996 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.479960    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nk7dm\" (UniqueName: \"kubernetes.io/projected/22d76305-7ecb-421f-9926-b1de7d91d2d8-kube-api-access-nk7dm\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:54.479996 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.479994    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22d76305-7ecb-421f-9926-b1de7d91d2d8-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:54.479996 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.480005    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22d76305-7ecb-421f-9926-b1de7d91d2d8-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:54.480288 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.480014    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22d76305-7ecb-421f-9926-b1de7d91d2d8-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:54.480288 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.480023    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22d76305-7ecb-421f-9926-b1de7d91d2d8-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:54.796794 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.796706    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:54.955473 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.955376    4871 generic.go:356] "Generic (PLEG): container finished" podID="22d76305-7ecb-421f-9926-b1de7d91d2d8" containerID="7375cf92a0a97662af605be144db34dda8698b0ef9952014400b80cd21e7c659" exitCode=0
Oct 30 22:18:54.955473 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.955427    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"
Oct 30 22:18:54.955473 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.955453    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj" event={"ID":"22d76305-7ecb-421f-9926-b1de7d91d2d8","Type":"ContainerDied","Data":"7375cf92a0a97662af605be144db34dda8698b0ef9952014400b80cd21e7c659"}
Oct 30 22:18:54.955752 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.955506    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj" event={"ID":"22d76305-7ecb-421f-9926-b1de7d91d2d8","Type":"ContainerDied","Data":"9c44e5e1193952642b5a183b4325c7c925c04f9708ef241ba1a3b21214f6fe6b"}
Oct 30 22:18:54.955752 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.955529    4871 scope.go:122] "RemoveContainer" containerID="7375cf92a0a97662af605be144db34dda8698b0ef9952014400b80cd21e7c659"
Oct 30 22:18:54.958849 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.958821    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb" event={"ID":"1fad116b-e1cc-42da-a439-bdecb46b96a9","Type":"ContainerStarted","Data":"6925013eb32b2bc795b436ed113177007fa8629b8f2b226802e10ab405000376"}
Oct 30 22:18:54.992882 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:54.992850    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:55.004772 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:55.004732    4871 scope.go:122] "RemoveContainer" containerID="5a883d0c34ef836dd4d3ec482f188166cbd34fdfaf1dbb97f50608ffff56f88d"
Oct 30 22:18:55.018214 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:55.018192    4871 scope.go:122] "RemoveContainer" containerID="7375cf92a0a97662af605be144db34dda8698b0ef9952014400b80cd21e7c659"
Oct 30 22:18:55.018602 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:18:55.018573    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7375cf92a0a97662af605be144db34dda8698b0ef9952014400b80cd21e7c659\": container with ID starting with 7375cf92a0a97662af605be144db34dda8698b0ef9952014400b80cd21e7c659 not found: ID does not exist" containerID="7375cf92a0a97662af605be144db34dda8698b0ef9952014400b80cd21e7c659"
Oct 30 22:18:55.018676 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:55.018615    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7375cf92a0a97662af605be144db34dda8698b0ef9952014400b80cd21e7c659"} err="failed to get container status \"7375cf92a0a97662af605be144db34dda8698b0ef9952014400b80cd21e7c659\": rpc error: code = NotFound desc = could not find container \"7375cf92a0a97662af605be144db34dda8698b0ef9952014400b80cd21e7c659\": container with ID starting with 7375cf92a0a97662af605be144db34dda8698b0ef9952014400b80cd21e7c659 not found: ID does not exist"
Oct 30 22:18:55.018676 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:55.018634    4871 scope.go:122] "RemoveContainer" containerID="5a883d0c34ef836dd4d3ec482f188166cbd34fdfaf1dbb97f50608ffff56f88d"
Oct 30 22:18:55.018951 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:18:55.018935    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a883d0c34ef836dd4d3ec482f188166cbd34fdfaf1dbb97f50608ffff56f88d\": container with ID starting with 5a883d0c34ef836dd4d3ec482f188166cbd34fdfaf1dbb97f50608ffff56f88d not found: ID does not exist" containerID="5a883d0c34ef836dd4d3ec482f188166cbd34fdfaf1dbb97f50608ffff56f88d"
Oct 30 22:18:55.019002 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:55.018963    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a883d0c34ef836dd4d3ec482f188166cbd34fdfaf1dbb97f50608ffff56f88d"} err="failed to get container status \"5a883d0c34ef836dd4d3ec482f188166cbd34fdfaf1dbb97f50608ffff56f88d\": rpc error: code = NotFound desc = could not find container \"5a883d0c34ef836dd4d3ec482f188166cbd34fdfaf1dbb97f50608ffff56f88d\": container with ID starting with 5a883d0c34ef836dd4d3ec482f188166cbd34fdfaf1dbb97f50608ffff56f88d not found: ID does not exist"
Oct 30 22:18:55.119620 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:55.119581    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"]
Oct 30 22:18:55.164327 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:55.164289    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-f0b8c9-openshift-default-85f478b987-txjkj"]
Oct 30 22:18:55.263585 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:55.263509    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"
Oct 30 22:18:55.439921 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:55.439880    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:55.966688 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:55.966650    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="22d76305-7ecb-421f-9926-b1de7d91d2d8" path="/var/lib/kubelet/pods/22d76305-7ecb-421f-9926-b1de7d91d2d8/volumes"
Oct 30 22:18:55.967600 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:55.967569    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="41ad0caa-9e1c-4dc1-9f70-43beb36de4a7" path="/var/lib/kubelet/pods/41ad0caa-9e1c-4dc1-9f70-43beb36de4a7/volumes"
Oct 30 22:18:55.972024 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:55.971992    4871 generic.go:356] "Generic (PLEG): container finished" podID="4a9121ce-2d96-4f01-8463-ef1768b5f588" containerID="600a071749ac5e0db389e71f1fc3831ab4e7ca9421ad5a984cf10b06d9f56705" exitCode=0
Oct 30 22:18:55.972164 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:55.972062    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4" event={"ID":"4a9121ce-2d96-4f01-8463-ef1768b5f588","Type":"ContainerDied","Data":"600a071749ac5e0db389e71f1fc3831ab4e7ca9421ad5a984cf10b06d9f56705"}
Oct 30 22:18:55.972164 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:55.972125    4871 scope.go:122] "RemoveContainer" containerID="25b482f2f890c986648d162aa82289d0f3e199d77e465e88f36e4bc5830d74fd"
Oct 30 22:18:55.974013 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:55.973991    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"
Oct 30 22:18:55.975057 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:55.975034    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"
Oct 30 22:18:56.055003 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:56.054977    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"
Oct 30 22:18:56.206072 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:56.206034    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4a9121ce-2d96-4f01-8463-ef1768b5f588-istio-token\" (UniqueName: \"kubernetes.io/projected/4a9121ce-2d96-4f01-8463-ef1768b5f588-istio-token\") pod \"4a9121ce-2d96-4f01-8463-ef1768b5f588\" (UID: \"4a9121ce-2d96-4f01-8463-ef1768b5f588\") "
Oct 30 22:18:56.206281 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:56.206098    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4a9121ce-2d96-4f01-8463-ef1768b5f588-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a9121ce-2d96-4f01-8463-ef1768b5f588-credential-socket\") pod \"4a9121ce-2d96-4f01-8463-ef1768b5f588\" (UID: \"4a9121ce-2d96-4f01-8463-ef1768b5f588\") "
Oct 30 22:18:56.206281 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:56.206124    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4a9121ce-2d96-4f01-8463-ef1768b5f588-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a9121ce-2d96-4f01-8463-ef1768b5f588-istio-data\") pod \"4a9121ce-2d96-4f01-8463-ef1768b5f588\" (UID: \"4a9121ce-2d96-4f01-8463-ef1768b5f588\") "
Oct 30 22:18:56.206281 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:56.206202    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4a9121ce-2d96-4f01-8463-ef1768b5f588-kube-api-access-vdlb5\" (UniqueName: \"kubernetes.io/projected/4a9121ce-2d96-4f01-8463-ef1768b5f588-kube-api-access-vdlb5\") pod \"4a9121ce-2d96-4f01-8463-ef1768b5f588\" (UID: \"4a9121ce-2d96-4f01-8463-ef1768b5f588\") "
Oct 30 22:18:56.206555 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:56.206285    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4a9121ce-2d96-4f01-8463-ef1768b5f588-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a9121ce-2d96-4f01-8463-ef1768b5f588-workload-certs\") pod \"4a9121ce-2d96-4f01-8463-ef1768b5f588\" (UID: \"4a9121ce-2d96-4f01-8463-ef1768b5f588\") "
Oct 30 22:18:56.206555 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:56.206349    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4a9121ce-2d96-4f01-8463-ef1768b5f588-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a9121ce-2d96-4f01-8463-ef1768b5f588-istio-envoy\") pod \"4a9121ce-2d96-4f01-8463-ef1768b5f588\" (UID: \"4a9121ce-2d96-4f01-8463-ef1768b5f588\") "
Oct 30 22:18:56.206555 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:56.206373    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4a9121ce-2d96-4f01-8463-ef1768b5f588-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a9121ce-2d96-4f01-8463-ef1768b5f588-istio-podinfo\") pod \"4a9121ce-2d96-4f01-8463-ef1768b5f588\" (UID: \"4a9121ce-2d96-4f01-8463-ef1768b5f588\") "
Oct 30 22:18:56.206555 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:56.206413    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4a9121ce-2d96-4f01-8463-ef1768b5f588-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a9121ce-2d96-4f01-8463-ef1768b5f588-istiod-ca-cert\") pod \"4a9121ce-2d96-4f01-8463-ef1768b5f588\" (UID: \"4a9121ce-2d96-4f01-8463-ef1768b5f588\") "
Oct 30 22:18:56.206555 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:56.206453    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4a9121ce-2d96-4f01-8463-ef1768b5f588-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a9121ce-2d96-4f01-8463-ef1768b5f588-workload-socket\") pod \"4a9121ce-2d96-4f01-8463-ef1768b5f588\" (UID: \"4a9121ce-2d96-4f01-8463-ef1768b5f588\") "
Oct 30 22:18:56.208564 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:56.207820    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a9121ce-2d96-4f01-8463-ef1768b5f588-workload-socket" pod "4a9121ce-2d96-4f01-8463-ef1768b5f588" (UID: "4a9121ce-2d96-4f01-8463-ef1768b5f588"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:56.208564 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:56.208088    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a9121ce-2d96-4f01-8463-ef1768b5f588-istio-data" pod "4a9121ce-2d96-4f01-8463-ef1768b5f588" (UID: "4a9121ce-2d96-4f01-8463-ef1768b5f588"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:56.231763 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:56.217993    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a9121ce-2d96-4f01-8463-ef1768b5f588-workload-certs" pod "4a9121ce-2d96-4f01-8463-ef1768b5f588" (UID: "4a9121ce-2d96-4f01-8463-ef1768b5f588"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:56.231763 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:56.218320    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a9121ce-2d96-4f01-8463-ef1768b5f588-istiod-ca-cert" pod "4a9121ce-2d96-4f01-8463-ef1768b5f588" (UID: "4a9121ce-2d96-4f01-8463-ef1768b5f588"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:18:56.231763 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:56.218759    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a9121ce-2d96-4f01-8463-ef1768b5f588-kube-api-access-vdlb5" pod "4a9121ce-2d96-4f01-8463-ef1768b5f588" (UID: "4a9121ce-2d96-4f01-8463-ef1768b5f588"). InnerVolumeSpecName "kube-api-access-vdlb5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:18:56.231763 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:56.224702    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a9121ce-2d96-4f01-8463-ef1768b5f588-istio-token" pod "4a9121ce-2d96-4f01-8463-ef1768b5f588" (UID: "4a9121ce-2d96-4f01-8463-ef1768b5f588"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:18:56.262962 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:56.234364    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a9121ce-2d96-4f01-8463-ef1768b5f588-credential-socket" pod "4a9121ce-2d96-4f01-8463-ef1768b5f588" (UID: "4a9121ce-2d96-4f01-8463-ef1768b5f588"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:56.303311 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:56.290404    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a9121ce-2d96-4f01-8463-ef1768b5f588-istio-envoy" pod "4a9121ce-2d96-4f01-8463-ef1768b5f588" (UID: "4a9121ce-2d96-4f01-8463-ef1768b5f588"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:56.333263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:56.329352    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a9121ce-2d96-4f01-8463-ef1768b5f588-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:56.333263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:56.329390    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a9121ce-2d96-4f01-8463-ef1768b5f588-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:56.333263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:56.329406    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a9121ce-2d96-4f01-8463-ef1768b5f588-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:56.333263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:56.329418    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a9121ce-2d96-4f01-8463-ef1768b5f588-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:56.333263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:56.329430    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a9121ce-2d96-4f01-8463-ef1768b5f588-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:56.333263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:56.329442    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vdlb5\" (UniqueName: \"kubernetes.io/projected/4a9121ce-2d96-4f01-8463-ef1768b5f588-kube-api-access-vdlb5\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:56.333263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:56.329462    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a9121ce-2d96-4f01-8463-ef1768b5f588-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:56.333263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:56.329475    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a9121ce-2d96-4f01-8463-ef1768b5f588-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:56.361434 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:56.358978    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4a9121ce-2d96-4f01-8463-ef1768b5f588-istio-podinfo" pod "4a9121ce-2d96-4f01-8463-ef1768b5f588" (UID: "4a9121ce-2d96-4f01-8463-ef1768b5f588"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:18:56.430643 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:56.430595    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a9121ce-2d96-4f01-8463-ef1768b5f588-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:18:56.941284 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:56.941229    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:56.982578 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:56.982546    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"
Oct 30 22:18:56.982780 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:56.982583    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4" event={"ID":"4a9121ce-2d96-4f01-8463-ef1768b5f588","Type":"ContainerDied","Data":"245f858eb2b097ceef474e1ef699e2f44278dd169f3fda48be7288ce66536ce5"}
Oct 30 22:18:56.982780 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:56.982638    4871 scope.go:122] "RemoveContainer" containerID="600a071749ac5e0db389e71f1fc3831ab4e7ca9421ad5a984cf10b06d9f56705"
Oct 30 22:18:57.153279 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:57.153219    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:57.197654 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:57.197562    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"]
Oct 30 22:18:57.205017 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:57.204977    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-138566-openshift-default-7f89f7669f-jlkr4"]
Oct 30 22:18:57.317220 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:57.317183    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:57.964722 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:18:57.964681    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4a9121ce-2d96-4f01-8463-ef1768b5f588" path="/var/lib/kubelet/pods/4a9121ce-2d96-4f01-8463-ef1768b5f588/volumes"
Oct 30 22:19:00.188398 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:00.188351    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:00.402116 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:00.400491    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:01.778549 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:01.778511    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:02.098138 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:02.098039    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:02.268027 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:02.267997    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:02.448123 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:02.448083    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"]
Oct 30 22:19:02.448589 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:02.448554    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk" podUID="78387fab-963c-4f00-9fda-33a5a5ebaee7" containerName="istio-proxy" containerID="cri-o://2cf5840cc20a141c7287efd9c4f75eaa7fd4ee5b9022ffc78dabb19676590bc5" gracePeriod=30
Oct 30 22:19:02.462923 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:02.462880    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:02.621473 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:02.621425    4871 patch_prober.go:29] interesting pod/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:19:02.621808 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:02.621781    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk" podUID="78387fab-963c-4f00-9fda-33a5a5ebaee7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:19:05.574073 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:19:05.574025    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:19:06.901019 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:06.900980    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:07.763439 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:07.763372    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"
Oct 30 22:19:07.853563 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:07.853514    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78387fab-963c-4f00-9fda-33a5a5ebaee7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78387fab-963c-4f00-9fda-33a5a5ebaee7-workload-socket\") pod \"78387fab-963c-4f00-9fda-33a5a5ebaee7\" (UID: \"78387fab-963c-4f00-9fda-33a5a5ebaee7\") "
Oct 30 22:19:07.853786 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:07.853582    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/78387fab-963c-4f00-9fda-33a5a5ebaee7-kube-api-access-zkp2m\" (UniqueName: \"kubernetes.io/projected/78387fab-963c-4f00-9fda-33a5a5ebaee7-kube-api-access-zkp2m\") pod \"78387fab-963c-4f00-9fda-33a5a5ebaee7\" (UID: \"78387fab-963c-4f00-9fda-33a5a5ebaee7\") "
Oct 30 22:19:07.853786 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:07.853634    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78387fab-963c-4f00-9fda-33a5a5ebaee7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78387fab-963c-4f00-9fda-33a5a5ebaee7-workload-certs\") pod \"78387fab-963c-4f00-9fda-33a5a5ebaee7\" (UID: \"78387fab-963c-4f00-9fda-33a5a5ebaee7\") "
Oct 30 22:19:07.853786 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:07.853679    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/78387fab-963c-4f00-9fda-33a5a5ebaee7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78387fab-963c-4f00-9fda-33a5a5ebaee7-istio-podinfo\") pod \"78387fab-963c-4f00-9fda-33a5a5ebaee7\" (UID: \"78387fab-963c-4f00-9fda-33a5a5ebaee7\") "
Oct 30 22:19:07.853786 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:07.853752    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78387fab-963c-4f00-9fda-33a5a5ebaee7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78387fab-963c-4f00-9fda-33a5a5ebaee7-istio-envoy\") pod \"78387fab-963c-4f00-9fda-33a5a5ebaee7\" (UID: \"78387fab-963c-4f00-9fda-33a5a5ebaee7\") "
Oct 30 22:19:07.854014 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:07.853813    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78387fab-963c-4f00-9fda-33a5a5ebaee7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78387fab-963c-4f00-9fda-33a5a5ebaee7-credential-socket\") pod \"78387fab-963c-4f00-9fda-33a5a5ebaee7\" (UID: \"78387fab-963c-4f00-9fda-33a5a5ebaee7\") "
Oct 30 22:19:07.854014 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:07.853855    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/78387fab-963c-4f00-9fda-33a5a5ebaee7-istio-token\" (UniqueName: \"kubernetes.io/projected/78387fab-963c-4f00-9fda-33a5a5ebaee7-istio-token\") pod \"78387fab-963c-4f00-9fda-33a5a5ebaee7\" (UID: \"78387fab-963c-4f00-9fda-33a5a5ebaee7\") "
Oct 30 22:19:07.854014 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:07.853866    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78387fab-963c-4f00-9fda-33a5a5ebaee7-workload-socket" pod "78387fab-963c-4f00-9fda-33a5a5ebaee7" (UID: "78387fab-963c-4f00-9fda-33a5a5ebaee7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:19:07.854014 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:07.853962    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/78387fab-963c-4f00-9fda-33a5a5ebaee7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78387fab-963c-4f00-9fda-33a5a5ebaee7-istiod-ca-cert\") pod \"78387fab-963c-4f00-9fda-33a5a5ebaee7\" (UID: \"78387fab-963c-4f00-9fda-33a5a5ebaee7\") "
Oct 30 22:19:07.854256 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:07.854020    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78387fab-963c-4f00-9fda-33a5a5ebaee7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78387fab-963c-4f00-9fda-33a5a5ebaee7-istio-data\") pod \"78387fab-963c-4f00-9fda-33a5a5ebaee7\" (UID: \"78387fab-963c-4f00-9fda-33a5a5ebaee7\") "
Oct 30 22:19:07.854256 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:07.854130    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78387fab-963c-4f00-9fda-33a5a5ebaee7-workload-certs" pod "78387fab-963c-4f00-9fda-33a5a5ebaee7" (UID: "78387fab-963c-4f00-9fda-33a5a5ebaee7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:19:07.854393 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:07.854318    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78387fab-963c-4f00-9fda-33a5a5ebaee7-credential-socket" pod "78387fab-963c-4f00-9fda-33a5a5ebaee7" (UID: "78387fab-963c-4f00-9fda-33a5a5ebaee7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:19:07.854445 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:07.854402    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78387fab-963c-4f00-9fda-33a5a5ebaee7-istio-data" pod "78387fab-963c-4f00-9fda-33a5a5ebaee7" (UID: "78387fab-963c-4f00-9fda-33a5a5ebaee7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:19:07.854504 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:07.854469    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78387fab-963c-4f00-9fda-33a5a5ebaee7-istiod-ca-cert" pod "78387fab-963c-4f00-9fda-33a5a5ebaee7" (UID: "78387fab-963c-4f00-9fda-33a5a5ebaee7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:19:07.854716 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:07.854689    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78387fab-963c-4f00-9fda-33a5a5ebaee7-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:07.854716 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:07.854716    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78387fab-963c-4f00-9fda-33a5a5ebaee7-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:07.854921 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:07.854725    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78387fab-963c-4f00-9fda-33a5a5ebaee7-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:07.854921 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:07.854734    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78387fab-963c-4f00-9fda-33a5a5ebaee7-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:07.854921 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:07.854743    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78387fab-963c-4f00-9fda-33a5a5ebaee7-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:07.856078 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:07.856053    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78387fab-963c-4f00-9fda-33a5a5ebaee7-kube-api-access-zkp2m" pod "78387fab-963c-4f00-9fda-33a5a5ebaee7" (UID: "78387fab-963c-4f00-9fda-33a5a5ebaee7"). InnerVolumeSpecName "kube-api-access-zkp2m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:19:07.856206 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:07.856122    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78387fab-963c-4f00-9fda-33a5a5ebaee7-istio-envoy" pod "78387fab-963c-4f00-9fda-33a5a5ebaee7" (UID: "78387fab-963c-4f00-9fda-33a5a5ebaee7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:19:07.856389 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:07.856371    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78387fab-963c-4f00-9fda-33a5a5ebaee7-istio-token" pod "78387fab-963c-4f00-9fda-33a5a5ebaee7" (UID: "78387fab-963c-4f00-9fda-33a5a5ebaee7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:19:07.856604 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:07.856589    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/78387fab-963c-4f00-9fda-33a5a5ebaee7-istio-podinfo" pod "78387fab-963c-4f00-9fda-33a5a5ebaee7" (UID: "78387fab-963c-4f00-9fda-33a5a5ebaee7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:19:07.955661 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:07.955619    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zkp2m\" (UniqueName: \"kubernetes.io/projected/78387fab-963c-4f00-9fda-33a5a5ebaee7-kube-api-access-zkp2m\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:07.955661 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:07.955664    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78387fab-963c-4f00-9fda-33a5a5ebaee7-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:07.956111 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:07.955680    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78387fab-963c-4f00-9fda-33a5a5ebaee7-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:07.956111 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:07.955695    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78387fab-963c-4f00-9fda-33a5a5ebaee7-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:08.113425 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:08.113390    4871 generic.go:356] "Generic (PLEG): container finished" podID="78387fab-963c-4f00-9fda-33a5a5ebaee7" containerID="2cf5840cc20a141c7287efd9c4f75eaa7fd4ee5b9022ffc78dabb19676590bc5" exitCode=0
Oct 30 22:19:08.113644 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:08.113486    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"
Oct 30 22:19:08.113644 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:08.113482    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk" event={"ID":"78387fab-963c-4f00-9fda-33a5a5ebaee7","Type":"ContainerDied","Data":"2cf5840cc20a141c7287efd9c4f75eaa7fd4ee5b9022ffc78dabb19676590bc5"}
Oct 30 22:19:08.113786 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:08.113740    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk" event={"ID":"78387fab-963c-4f00-9fda-33a5a5ebaee7","Type":"ContainerDied","Data":"f2bbca7b2ba5bebb83f7d1ed044f4b494a30fa84c0584d55d1f7686f9ffbe30e"}
Oct 30 22:19:08.113847 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:08.113804    4871 scope.go:122] "RemoveContainer" containerID="2cf5840cc20a141c7287efd9c4f75eaa7fd4ee5b9022ffc78dabb19676590bc5"
Oct 30 22:19:08.116478 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:08.116455    4871 generic.go:356] "Generic (PLEG): container finished" podID="cf4abc58-fe24-486b-a195-1eb2a3b2e643" containerID="0cbc0d71509db4413284c81b3a059c69a31415cd63a8b165fe83724ac10162f9" exitCode=0
Oct 30 22:19:08.116630 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:08.116501    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2" event={"ID":"cf4abc58-fe24-486b-a195-1eb2a3b2e643","Type":"ContainerDied","Data":"0cbc0d71509db4413284c81b3a059c69a31415cd63a8b165fe83724ac10162f9"}
Oct 30 22:19:08.116956 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:08.116927    4871 scope.go:122] "RemoveContainer" containerID="0cbc0d71509db4413284c81b3a059c69a31415cd63a8b165fe83724ac10162f9"
Oct 30 22:19:08.118871 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:08.118830    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:19:08.118995 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:08.118930    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:19:08.119061 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:08.118977    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:19:08.126168 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:08.126145    4871 scope.go:122] "RemoveContainer" containerID="dd37dcafa918dc0f028207aebe5fd1fd38fca109ef8ab1c431aa6ca3b78ed693"
Oct 30 22:19:08.141148 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:08.141123    4871 scope.go:122] "RemoveContainer" containerID="2cf5840cc20a141c7287efd9c4f75eaa7fd4ee5b9022ffc78dabb19676590bc5"
Oct 30 22:19:08.141565 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:19:08.141539    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cf5840cc20a141c7287efd9c4f75eaa7fd4ee5b9022ffc78dabb19676590bc5\": container with ID starting with 2cf5840cc20a141c7287efd9c4f75eaa7fd4ee5b9022ffc78dabb19676590bc5 not found: ID does not exist" containerID="2cf5840cc20a141c7287efd9c4f75eaa7fd4ee5b9022ffc78dabb19676590bc5"
Oct 30 22:19:08.141675 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:08.141579    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cf5840cc20a141c7287efd9c4f75eaa7fd4ee5b9022ffc78dabb19676590bc5"} err="failed to get container status \"2cf5840cc20a141c7287efd9c4f75eaa7fd4ee5b9022ffc78dabb19676590bc5\": rpc error: code = NotFound desc = could not find container \"2cf5840cc20a141c7287efd9c4f75eaa7fd4ee5b9022ffc78dabb19676590bc5\": container with ID starting with 2cf5840cc20a141c7287efd9c4f75eaa7fd4ee5b9022ffc78dabb19676590bc5 not found: ID does not exist"
Oct 30 22:19:08.141675 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:08.141605    4871 scope.go:122] "RemoveContainer" containerID="dd37dcafa918dc0f028207aebe5fd1fd38fca109ef8ab1c431aa6ca3b78ed693"
Oct 30 22:19:08.141974 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:19:08.141946    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd37dcafa918dc0f028207aebe5fd1fd38fca109ef8ab1c431aa6ca3b78ed693\": container with ID starting with dd37dcafa918dc0f028207aebe5fd1fd38fca109ef8ab1c431aa6ca3b78ed693 not found: ID does not exist" containerID="dd37dcafa918dc0f028207aebe5fd1fd38fca109ef8ab1c431aa6ca3b78ed693"
Oct 30 22:19:08.142027 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:08.141983    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd37dcafa918dc0f028207aebe5fd1fd38fca109ef8ab1c431aa6ca3b78ed693"} err="failed to get container status \"dd37dcafa918dc0f028207aebe5fd1fd38fca109ef8ab1c431aa6ca3b78ed693\": rpc error: code = NotFound desc = could not find container \"dd37dcafa918dc0f028207aebe5fd1fd38fca109ef8ab1c431aa6ca3b78ed693\": container with ID starting with dd37dcafa918dc0f028207aebe5fd1fd38fca109ef8ab1c431aa6ca3b78ed693 not found: ID does not exist"
Oct 30 22:19:08.388458 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:08.388356    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"]
Oct 30 22:19:08.392077 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:08.392042    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-65659d-openshift-default-7b95dbfbf7-87stk"]
Oct 30 22:19:09.137157 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:09.137115    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2" event={"ID":"cf4abc58-fe24-486b-a195-1eb2a3b2e643","Type":"ContainerStarted","Data":"19afab03874a37cb1c0466fa9cecc8a14a2b2e90573756821cfa903f027967a1"}
Oct 30 22:19:09.194065 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:09.194028    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"
Oct 30 22:19:09.194273 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:09.194222    4871 patch_prober.go:29] interesting pod/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused" start-of-body=
Oct 30 22:19:09.194347 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:09.194282    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2" podUID="cf4abc58-fe24-486b-a195-1eb2a3b2e643" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused"
Oct 30 22:19:09.960683 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:09.960646    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="78387fab-963c-4f00-9fda-33a5a5ebaee7" path="/var/lib/kubelet/pods/78387fab-963c-4f00-9fda-33a5a5ebaee7/volumes"
Oct 30 22:19:10.193948 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:10.193909    4871 patch_prober.go:29] interesting pod/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused" start-of-body=
Oct 30 22:19:10.194418 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:10.193981    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2" podUID="cf4abc58-fe24-486b-a195-1eb2a3b2e643" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused"
Oct 30 22:19:10.927901 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:10.927859    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274"]
Oct 30 22:19:10.928217 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:10.928139    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274" podUID="16ac9cf9-cfd8-42dd-9a72-094ff81da9b2" containerName="maas-api" containerID="cri-o://e8d016837e1d5cc59474a114f42449a09a528a07ccccaa649826d39b3d9ddd53" gracePeriod=30
Oct 30 22:19:11.032216 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:11.032143    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:11.226719 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:11.200667    4871 patch_prober.go:29] interesting pod/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused" start-of-body=
Oct 30 22:19:11.226719 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:11.200734    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2" podUID="cf4abc58-fe24-486b-a195-1eb2a3b2e643" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused"
Oct 30 22:19:11.243945 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:11.243912    4871 generic.go:356] "Generic (PLEG): container finished" podID="16ac9cf9-cfd8-42dd-9a72-094ff81da9b2" containerID="e8d016837e1d5cc59474a114f42449a09a528a07ccccaa649826d39b3d9ddd53" exitCode=0
Oct 30 22:19:11.244181 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:11.243981    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274" event={"ID":"16ac9cf9-cfd8-42dd-9a72-094ff81da9b2","Type":"ContainerDied","Data":"e8d016837e1d5cc59474a114f42449a09a528a07ccccaa649826d39b3d9ddd53"}
Oct 30 22:19:11.282463 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:11.282434    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274"
Oct 30 22:19:11.319320 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:11.307772    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/16ac9cf9-cfd8-42dd-9a72-094ff81da9b2-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16ac9cf9-cfd8-42dd-9a72-094ff81da9b2-maas-api-tls\") pod \"16ac9cf9-cfd8-42dd-9a72-094ff81da9b2\" (UID: \"16ac9cf9-cfd8-42dd-9a72-094ff81da9b2\") "
Oct 30 22:19:11.319320 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:11.307850    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/16ac9cf9-cfd8-42dd-9a72-094ff81da9b2-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16ac9cf9-cfd8-42dd-9a72-094ff81da9b2-openshift-service-ca\") pod \"16ac9cf9-cfd8-42dd-9a72-094ff81da9b2\" (UID: \"16ac9cf9-cfd8-42dd-9a72-094ff81da9b2\") "
Oct 30 22:19:11.319320 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:11.307959    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/16ac9cf9-cfd8-42dd-9a72-094ff81da9b2-kube-api-access-gqcsl\" (UniqueName: \"kubernetes.io/projected/16ac9cf9-cfd8-42dd-9a72-094ff81da9b2-kube-api-access-gqcsl\") pod \"16ac9cf9-cfd8-42dd-9a72-094ff81da9b2\" (UID: \"16ac9cf9-cfd8-42dd-9a72-094ff81da9b2\") "
Oct 30 22:19:11.319320 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:11.308002    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/16ac9cf9-cfd8-42dd-9a72-094ff81da9b2-metrics-certs\" (UniqueName: \"kubernetes.io/secret/16ac9cf9-cfd8-42dd-9a72-094ff81da9b2-metrics-certs\") pod \"16ac9cf9-cfd8-42dd-9a72-094ff81da9b2\" (UID: \"16ac9cf9-cfd8-42dd-9a72-094ff81da9b2\") "
Oct 30 22:19:11.319320 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:11.308342    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16ac9cf9-cfd8-42dd-9a72-094ff81da9b2-openshift-service-ca" pod "16ac9cf9-cfd8-42dd-9a72-094ff81da9b2" (UID: "16ac9cf9-cfd8-42dd-9a72-094ff81da9b2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:19:11.319320 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:11.309128    4871 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16ac9cf9-cfd8-42dd-9a72-094ff81da9b2-openshift-service-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:11.367588 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:11.365603    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16ac9cf9-cfd8-42dd-9a72-094ff81da9b2-maas-api-tls" pod "16ac9cf9-cfd8-42dd-9a72-094ff81da9b2" (UID: "16ac9cf9-cfd8-42dd-9a72-094ff81da9b2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:19:11.367588 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:11.366166    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16ac9cf9-cfd8-42dd-9a72-094ff81da9b2-metrics-certs" pod "16ac9cf9-cfd8-42dd-9a72-094ff81da9b2" (UID: "16ac9cf9-cfd8-42dd-9a72-094ff81da9b2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:19:11.367588 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:11.366278    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16ac9cf9-cfd8-42dd-9a72-094ff81da9b2-kube-api-access-gqcsl" pod "16ac9cf9-cfd8-42dd-9a72-094ff81da9b2" (UID: "16ac9cf9-cfd8-42dd-9a72-094ff81da9b2"). InnerVolumeSpecName "kube-api-access-gqcsl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:19:11.413252 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:11.413135    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16ac9cf9-cfd8-42dd-9a72-094ff81da9b2-maas-api-tls\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:11.413252 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:11.413181    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gqcsl\" (UniqueName: \"kubernetes.io/projected/16ac9cf9-cfd8-42dd-9a72-094ff81da9b2-kube-api-access-gqcsl\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:11.413252 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:11.413198    4871 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16ac9cf9-cfd8-42dd-9a72-094ff81da9b2-metrics-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:11.571020 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:11.570912    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:11.724262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:11.723876    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:12.145248 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:12.145197    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4"]
Oct 30 22:19:12.145448 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:12.145424    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4" podUID="fe4334e5-e953-4fdb-901d-1350c70012f1" containerName="maas-api" containerID="cri-o://1a6dc0a4e3a5c6425556f270a880a19ff7125e9aaf83e34c88cba168d3ecfb9e" gracePeriod=30
Oct 30 22:19:12.196753 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:12.196715    4871 patch_prober.go:29] interesting pod/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused" start-of-body=
Oct 30 22:19:12.196957 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:12.196783    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2" podUID="cf4abc58-fe24-486b-a195-1eb2a3b2e643" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused"
Oct 30 22:19:12.253722 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:12.253682    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274"
Oct 30 22:19:12.253722 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:12.253712    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274" event={"ID":"16ac9cf9-cfd8-42dd-9a72-094ff81da9b2","Type":"ContainerDied","Data":"9639fa0288e46ccdc968a72d60eb89d88562d83d59e38a51d1d3db4c8a9c9599"}
Oct 30 22:19:12.254213 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:12.253781    4871 scope.go:122] "RemoveContainer" containerID="e8d016837e1d5cc59474a114f42449a09a528a07ccccaa649826d39b3d9ddd53"
Oct 30 22:19:12.286632 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:12.286593    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:12.403263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:12.401937    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274"]
Oct 30 22:19:12.447373 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:12.447135    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4bfac-9b68c7944-64274"]
Oct 30 22:19:12.491824 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:12.491792    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4"
Oct 30 22:19:12.625928 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:12.625888    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:12.650125 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:12.635007    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fe4334e5-e953-4fdb-901d-1350c70012f1-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe4334e5-e953-4fdb-901d-1350c70012f1-maas-api-tls\") pod \"fe4334e5-e953-4fdb-901d-1350c70012f1\" (UID: \"fe4334e5-e953-4fdb-901d-1350c70012f1\") "
Oct 30 22:19:12.650125 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:12.645722    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fe4334e5-e953-4fdb-901d-1350c70012f1-kube-api-access-xtwql\" (UniqueName: \"kubernetes.io/projected/fe4334e5-e953-4fdb-901d-1350c70012f1-kube-api-access-xtwql\") pod \"fe4334e5-e953-4fdb-901d-1350c70012f1\" (UID: \"fe4334e5-e953-4fdb-901d-1350c70012f1\") "
Oct 30 22:19:12.650125 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:12.645836    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fe4334e5-e953-4fdb-901d-1350c70012f1-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe4334e5-e953-4fdb-901d-1350c70012f1-openshift-service-ca\") pod \"fe4334e5-e953-4fdb-901d-1350c70012f1\" (UID: \"fe4334e5-e953-4fdb-901d-1350c70012f1\") "
Oct 30 22:19:12.650125 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:12.645958    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fe4334e5-e953-4fdb-901d-1350c70012f1-metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe4334e5-e953-4fdb-901d-1350c70012f1-metrics-certs\") pod \"fe4334e5-e953-4fdb-901d-1350c70012f1\" (UID: \"fe4334e5-e953-4fdb-901d-1350c70012f1\") "
Oct 30 22:19:12.650125 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:12.641265    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe4334e5-e953-4fdb-901d-1350c70012f1-maas-api-tls" pod "fe4334e5-e953-4fdb-901d-1350c70012f1" (UID: "fe4334e5-e953-4fdb-901d-1350c70012f1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:19:12.650125 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:12.646687    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe4334e5-e953-4fdb-901d-1350c70012f1-maas-api-tls\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:12.652053 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:12.652008    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe4334e5-e953-4fdb-901d-1350c70012f1-openshift-service-ca" pod "fe4334e5-e953-4fdb-901d-1350c70012f1" (UID: "fe4334e5-e953-4fdb-901d-1350c70012f1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:19:12.653475 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:12.653421    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe4334e5-e953-4fdb-901d-1350c70012f1-kube-api-access-xtwql" pod "fe4334e5-e953-4fdb-901d-1350c70012f1" (UID: "fe4334e5-e953-4fdb-901d-1350c70012f1"). InnerVolumeSpecName "kube-api-access-xtwql". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:19:12.655344 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:12.655310    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe4334e5-e953-4fdb-901d-1350c70012f1-metrics-certs" pod "fe4334e5-e953-4fdb-901d-1350c70012f1" (UID: "fe4334e5-e953-4fdb-901d-1350c70012f1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:19:12.748665 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:12.748621    4871 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe4334e5-e953-4fdb-901d-1350c70012f1-metrics-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:12.748665 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:12.748669    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtwql\" (UniqueName: \"kubernetes.io/projected/fe4334e5-e953-4fdb-901d-1350c70012f1-kube-api-access-xtwql\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:12.748875 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:12.748686    4871 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe4334e5-e953-4fdb-901d-1350c70012f1-openshift-service-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:12.827975 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:12.827882    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:12.980691 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:12.980576    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:13.207285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:13.197859    4871 patch_prober.go:29] interesting pod/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused" start-of-body=
Oct 30 22:19:13.207285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:13.197944    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2" podUID="cf4abc58-fe24-486b-a195-1eb2a3b2e643" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused"
Oct 30 22:19:13.266937 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:13.266841    4871 generic.go:356] "Generic (PLEG): container finished" podID="fe4334e5-e953-4fdb-901d-1350c70012f1" containerID="1a6dc0a4e3a5c6425556f270a880a19ff7125e9aaf83e34c88cba168d3ecfb9e" exitCode=0
Oct 30 22:19:13.267342 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:13.266941    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4"
Oct 30 22:19:13.267342 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:13.266964    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4" event={"ID":"fe4334e5-e953-4fdb-901d-1350c70012f1","Type":"ContainerDied","Data":"1a6dc0a4e3a5c6425556f270a880a19ff7125e9aaf83e34c88cba168d3ecfb9e"}
Oct 30 22:19:13.267342 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:13.267028    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4" event={"ID":"fe4334e5-e953-4fdb-901d-1350c70012f1","Type":"ContainerDied","Data":"c8cc7a52890392b71536c0b07cc7381753333d56c95092560f35a73d94319e0e"}
Oct 30 22:19:13.267342 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:13.267063    4871 scope.go:122] "RemoveContainer" containerID="1a6dc0a4e3a5c6425556f270a880a19ff7125e9aaf83e34c88cba168d3ecfb9e"
Oct 30 22:19:13.282214 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:13.282188    4871 scope.go:122] "RemoveContainer" containerID="1a6dc0a4e3a5c6425556f270a880a19ff7125e9aaf83e34c88cba168d3ecfb9e"
Oct 30 22:19:13.282611 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:19:13.282589    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a6dc0a4e3a5c6425556f270a880a19ff7125e9aaf83e34c88cba168d3ecfb9e\": container with ID starting with 1a6dc0a4e3a5c6425556f270a880a19ff7125e9aaf83e34c88cba168d3ecfb9e not found: ID does not exist" containerID="1a6dc0a4e3a5c6425556f270a880a19ff7125e9aaf83e34c88cba168d3ecfb9e"
Oct 30 22:19:13.282686 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:13.282621    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a6dc0a4e3a5c6425556f270a880a19ff7125e9aaf83e34c88cba168d3ecfb9e"} err="failed to get container status \"1a6dc0a4e3a5c6425556f270a880a19ff7125e9aaf83e34c88cba168d3ecfb9e\": rpc error: code = NotFound desc = could not find container \"1a6dc0a4e3a5c6425556f270a880a19ff7125e9aaf83e34c88cba168d3ecfb9e\": container with ID starting with 1a6dc0a4e3a5c6425556f270a880a19ff7125e9aaf83e34c88cba168d3ecfb9e not found: ID does not exist"
Oct 30 22:19:13.444786 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:13.444739    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4"]
Oct 30 22:19:13.447906 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:13.447870    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5d2b13-b6b576cc8-h4bl4"]
Oct 30 22:19:13.562551 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:13.562463    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:13.961406 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:13.961363    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="16ac9cf9-cfd8-42dd-9a72-094ff81da9b2" path="/var/lib/kubelet/pods/16ac9cf9-cfd8-42dd-9a72-094ff81da9b2/volumes"
Oct 30 22:19:13.961919 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:13.961900    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fe4334e5-e953-4fdb-901d-1350c70012f1" path="/var/lib/kubelet/pods/fe4334e5-e953-4fdb-901d-1350c70012f1/volumes"
Oct 30 22:19:14.024263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:14.020352    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"
Oct 30 22:19:14.162088 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:14.162050    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:14.194694 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:14.194645    4871 patch_prober.go:29] interesting pod/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused" start-of-body=
Oct 30 22:19:14.195085 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:14.195026    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2" podUID="cf4abc58-fe24-486b-a195-1eb2a3b2e643" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused"
Oct 30 22:19:14.323582 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:14.323464    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:14.472147 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:14.472110    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:15.004564 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:15.004528    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:15.153643 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:15.153605    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:15.198851 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:15.198819    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"
Oct 30 22:19:15.199952 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:15.199930    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"
Oct 30 22:19:15.982097 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:15.982062    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:19.200274 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:19.200214    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:19.461828 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:19.461732    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:19.616973 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:19.616917    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:19.795971 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:19.795894    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"]
Oct 30 22:19:19.796366 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:19.796332    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856" podUID="6674b9d4-017b-4109-9262-bc8ad1244712" containerName="istio-proxy" containerID="cri-o://da178ee02a1013c7b9cfc72cd53ca60dbe18900ff5139ea376e4466cc1b24469" gracePeriod=30
Oct 30 22:19:19.838255 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:19.838203    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:19.993203 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:19.993171    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:21.365171 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:21.365138    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:21.633892 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:21.633796    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:21.807125 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:21.807079    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:21.937247 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:21.937145    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"]
Oct 30 22:19:21.938088 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:21.938063    4871 cpu_manager.go:414] "containerMap: removing container" podUID="22d76305-7ecb-421f-9926-b1de7d91d2d8" containerName="istio-proxy"
Oct 30 22:19:21.938288 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:21.938263    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="22d76305-7ecb-421f-9926-b1de7d91d2d8" containerName="istio-proxy"
Oct 30 22:19:21.938438 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:21.938294    4871 cpu_manager.go:414] "containerMap: removing container" podUID="fe4334e5-e953-4fdb-901d-1350c70012f1" containerName="maas-api"
Oct 30 22:19:21.938438 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:21.938302    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fe4334e5-e953-4fdb-901d-1350c70012f1" containerName="maas-api"
Oct 30 22:19:21.938438 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:21.938328    4871 cpu_manager.go:414] "containerMap: removing container" podUID="78387fab-963c-4f00-9fda-33a5a5ebaee7" containerName="istio-proxy"
Oct 30 22:19:21.938438 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:21.938336    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="78387fab-963c-4f00-9fda-33a5a5ebaee7" containerName="istio-proxy"
Oct 30 22:19:21.938438 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:21.938347    4871 cpu_manager.go:414] "containerMap: removing container" podUID="16ac9cf9-cfd8-42dd-9a72-094ff81da9b2" containerName="maas-api"
Oct 30 22:19:21.938438 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:21.938355    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="16ac9cf9-cfd8-42dd-9a72-094ff81da9b2" containerName="maas-api"
Oct 30 22:19:21.938438 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:21.938372    4871 cpu_manager.go:414] "containerMap: removing container" podUID="22d76305-7ecb-421f-9926-b1de7d91d2d8" containerName="istio-proxy"
Oct 30 22:19:21.938438 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:21.938379    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="22d76305-7ecb-421f-9926-b1de7d91d2d8" containerName="istio-proxy"
Oct 30 22:19:21.938438 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:21.938413    4871 cpu_manager.go:414] "containerMap: removing container" podUID="4a9121ce-2d96-4f01-8463-ef1768b5f588" containerName="istio-proxy"
Oct 30 22:19:21.938438 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:21.938422    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4a9121ce-2d96-4f01-8463-ef1768b5f588" containerName="istio-proxy"
Oct 30 22:19:21.938984 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:21.938449    4871 cpu_manager.go:414] "containerMap: removing container" podUID="41ad0caa-9e1c-4dc1-9f70-43beb36de4a7" containerName="maas-api"
Oct 30 22:19:21.938984 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:21.938457    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="41ad0caa-9e1c-4dc1-9f70-43beb36de4a7" containerName="maas-api"
Oct 30 22:19:21.938984 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:21.938617    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="4a9121ce-2d96-4f01-8463-ef1768b5f588" containerName="istio-proxy"
Oct 30 22:19:21.938984 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:21.938631    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="41ad0caa-9e1c-4dc1-9f70-43beb36de4a7" containerName="maas-api"
Oct 30 22:19:21.938984 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:21.938644    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="22d76305-7ecb-421f-9926-b1de7d91d2d8" containerName="istio-proxy"
Oct 30 22:19:21.938984 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:21.938654    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="fe4334e5-e953-4fdb-901d-1350c70012f1" containerName="maas-api"
Oct 30 22:19:21.938984 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:21.938664    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="78387fab-963c-4f00-9fda-33a5a5ebaee7" containerName="istio-proxy"
Oct 30 22:19:21.938984 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:21.938672    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="16ac9cf9-cfd8-42dd-9a72-094ff81da9b2" containerName="maas-api"
Oct 30 22:19:21.938984 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:21.938680    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="22d76305-7ecb-421f-9926-b1de7d91d2d8" containerName="istio-proxy"
Oct 30 22:19:21.938984 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:21.938690    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="78387fab-963c-4f00-9fda-33a5a5ebaee7" containerName="istio-proxy"
Oct 30 22:19:21.944643 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:21.944615    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"
Oct 30 22:19:21.982957 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:21.982915    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d334339-1d4b-421b-856e-c2a0d9f4737a-workload-socket\") pod \"e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g\" (UID: \"4d334339-1d4b-421b-856e-c2a0d9f4737a\") " pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"
Oct 30 22:19:21.983175 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:21.982985    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d334339-1d4b-421b-856e-c2a0d9f4737a-istio-data\") pod \"e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g\" (UID: \"4d334339-1d4b-421b-856e-c2a0d9f4737a\") " pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"
Oct 30 22:19:21.983270 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:21.983218    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d334339-1d4b-421b-856e-c2a0d9f4737a-credential-socket\") pod \"e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g\" (UID: \"4d334339-1d4b-421b-856e-c2a0d9f4737a\") " pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"
Oct 30 22:19:21.983344 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:21.983290    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6fzx\" (UniqueName: \"kubernetes.io/projected/4d334339-1d4b-421b-856e-c2a0d9f4737a-kube-api-access-f6fzx\") pod \"e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g\" (UID: \"4d334339-1d4b-421b-856e-c2a0d9f4737a\") " pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"
Oct 30 22:19:21.983344 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:21.983321    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d334339-1d4b-421b-856e-c2a0d9f4737a-istiod-ca-cert\") pod \"e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g\" (UID: \"4d334339-1d4b-421b-856e-c2a0d9f4737a\") " pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"
Oct 30 22:19:21.983556 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:21.983530    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d334339-1d4b-421b-856e-c2a0d9f4737a-istio-envoy\") pod \"e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g\" (UID: \"4d334339-1d4b-421b-856e-c2a0d9f4737a\") " pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"
Oct 30 22:19:21.983659 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:21.983635    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d334339-1d4b-421b-856e-c2a0d9f4737a-istio-token\") pod \"e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g\" (UID: \"4d334339-1d4b-421b-856e-c2a0d9f4737a\") " pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"
Oct 30 22:19:21.983727 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:21.983696    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d334339-1d4b-421b-856e-c2a0d9f4737a-istio-podinfo\") pod \"e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g\" (UID: \"4d334339-1d4b-421b-856e-c2a0d9f4737a\") " pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"
Oct 30 22:19:21.983816 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:21.983795    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d334339-1d4b-421b-856e-c2a0d9f4737a-workload-certs\") pod \"e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g\" (UID: \"4d334339-1d4b-421b-856e-c2a0d9f4737a\") " pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"
Oct 30 22:19:21.995924 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:21.995889    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-a-ca6d0b-openshift-default-dockercfg-tp7lc\""
Oct 30 22:19:22.018674 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:22.018635    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:22.051300 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:22.051257    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"]
Oct 30 22:19:22.090536 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:22.090477    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d334339-1d4b-421b-856e-c2a0d9f4737a-istio-data\") pod \"e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g\" (UID: \"4d334339-1d4b-421b-856e-c2a0d9f4737a\") " pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"
Oct 30 22:19:22.090747 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:22.090596    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d334339-1d4b-421b-856e-c2a0d9f4737a-credential-socket\") pod \"e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g\" (UID: \"4d334339-1d4b-421b-856e-c2a0d9f4737a\") " pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"
Oct 30 22:19:22.090747 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:22.090632    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6fzx\" (UniqueName: \"kubernetes.io/projected/4d334339-1d4b-421b-856e-c2a0d9f4737a-kube-api-access-f6fzx\") pod \"e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g\" (UID: \"4d334339-1d4b-421b-856e-c2a0d9f4737a\") " pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"
Oct 30 22:19:22.090747 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:22.090659    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d334339-1d4b-421b-856e-c2a0d9f4737a-istiod-ca-cert\") pod \"e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g\" (UID: \"4d334339-1d4b-421b-856e-c2a0d9f4737a\") " pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"
Oct 30 22:19:22.090920 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:22.090763    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d334339-1d4b-421b-856e-c2a0d9f4737a-istio-envoy\") pod \"e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g\" (UID: \"4d334339-1d4b-421b-856e-c2a0d9f4737a\") " pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"
Oct 30 22:19:22.090920 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:22.090832    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d334339-1d4b-421b-856e-c2a0d9f4737a-istio-token\") pod \"e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g\" (UID: \"4d334339-1d4b-421b-856e-c2a0d9f4737a\") " pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"
Oct 30 22:19:22.090920 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:22.090903    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d334339-1d4b-421b-856e-c2a0d9f4737a-istio-podinfo\") pod \"e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g\" (UID: \"4d334339-1d4b-421b-856e-c2a0d9f4737a\") " pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"
Oct 30 22:19:22.091082 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:22.090943    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d334339-1d4b-421b-856e-c2a0d9f4737a-workload-certs\") pod \"e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g\" (UID: \"4d334339-1d4b-421b-856e-c2a0d9f4737a\") " pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"
Oct 30 22:19:22.091082 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:22.091013    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d334339-1d4b-421b-856e-c2a0d9f4737a-workload-socket\") pod \"e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g\" (UID: \"4d334339-1d4b-421b-856e-c2a0d9f4737a\") " pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"
Oct 30 22:19:22.130514 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:22.116706    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d334339-1d4b-421b-856e-c2a0d9f4737a-workload-socket\") pod \"e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g\" (UID: \"4d334339-1d4b-421b-856e-c2a0d9f4737a\") " pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"
Oct 30 22:19:22.130514 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:22.127301    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d334339-1d4b-421b-856e-c2a0d9f4737a-credential-socket\") pod \"e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g\" (UID: \"4d334339-1d4b-421b-856e-c2a0d9f4737a\") " pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"
Oct 30 22:19:22.130514 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:22.128545    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d334339-1d4b-421b-856e-c2a0d9f4737a-istiod-ca-cert\") pod \"e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g\" (UID: \"4d334339-1d4b-421b-856e-c2a0d9f4737a\") " pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"
Oct 30 22:19:22.130514 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:22.129111    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d334339-1d4b-421b-856e-c2a0d9f4737a-workload-certs\") pod \"e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g\" (UID: \"4d334339-1d4b-421b-856e-c2a0d9f4737a\") " pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"
Oct 30 22:19:22.130514 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:22.129535    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d334339-1d4b-421b-856e-c2a0d9f4737a-istio-data\") pod \"e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g\" (UID: \"4d334339-1d4b-421b-856e-c2a0d9f4737a\") " pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"
Oct 30 22:19:22.172804 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:22.172739    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d334339-1d4b-421b-856e-c2a0d9f4737a-istio-envoy\") pod \"e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g\" (UID: \"4d334339-1d4b-421b-856e-c2a0d9f4737a\") " pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"
Oct 30 22:19:22.173063 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:22.173035    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d334339-1d4b-421b-856e-c2a0d9f4737a-istio-podinfo\") pod \"e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g\" (UID: \"4d334339-1d4b-421b-856e-c2a0d9f4737a\") " pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"
Oct 30 22:19:22.180570 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:22.180537    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d334339-1d4b-421b-856e-c2a0d9f4737a-istio-token\") pod \"e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g\" (UID: \"4d334339-1d4b-421b-856e-c2a0d9f4737a\") " pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"
Oct 30 22:19:22.186382 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:22.186346    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:22.190109 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:22.190024    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6fzx\" (UniqueName: \"kubernetes.io/projected/4d334339-1d4b-421b-856e-c2a0d9f4737a-kube-api-access-f6fzx\") pod \"e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g\" (UID: \"4d334339-1d4b-421b-856e-c2a0d9f4737a\") " pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"
Oct 30 22:19:22.272169 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:22.272129    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"
Oct 30 22:19:22.353669 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:22.353625    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:22.703328 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:22.703297    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"]
Oct 30 22:19:22.704749 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:19:22.704707    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d334339_1d4b_421b_856e_c2a0d9f4737a.slice/crio-7ed71ac2510dba28351383f4713b939f07ac53b29548f57bd9ae156994c0ad41 WatchSource:0}: Error finding container 7ed71ac2510dba28351383f4713b939f07ac53b29548f57bd9ae156994c0ad41: Status 404 returned error can't find the container with id 7ed71ac2510dba28351383f4713b939f07ac53b29548f57bd9ae156994c0ad41
Oct 30 22:19:22.708100 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:22.708056    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:19:22.708209 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:22.708130    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:19:22.708209 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:22.708163    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:19:23.069943 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:23.069846    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:23.368616 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:23.368571    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g" event={"ID":"4d334339-1d4b-421b-856e-c2a0d9f4737a","Type":"ContainerStarted","Data":"ad7e4c29bbccf337b323631161bab60909fe00c53884b029ad4567f3ea592ef6"}
Oct 30 22:19:23.368831 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:23.368670    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g" event={"ID":"4d334339-1d4b-421b-856e-c2a0d9f4737a","Type":"ContainerStarted","Data":"7ed71ac2510dba28351383f4713b939f07ac53b29548f57bd9ae156994c0ad41"}
Oct 30 22:19:23.468292 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:23.468256    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-63a282-5978dfffdf-wtlnf"]
Oct 30 22:19:23.533534 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:23.533465    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g" podStartSLOduration=2.533437512 podStartE2EDuration="2.533437512s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:19:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:19:23.530090299 +0000 UTC m=+2505.707578918" watchObservedRunningTime="2026-10-30 22:19:23.533437512 +0000 UTC m=+2505.710926193"
Oct 30 22:19:23.570343 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:23.570300    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-63a282-6c49469979-hf22h"]
Oct 30 22:19:23.595629 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:23.595595    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:23.849698 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:23.849599    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:24.002444 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.002406    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:24.215606 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.215569    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:24.276294 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.273543    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"
Oct 30 22:19:24.286549 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.286508    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"
Oct 30 22:19:24.376475 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.376446    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"
Oct 30 22:19:24.377419 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.377399    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"
Oct 30 22:19:24.620420 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.620388    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"]
Oct 30 22:19:24.621167 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.621129    4871 cpu_manager.go:414] "containerMap: removing container" podUID="4a9121ce-2d96-4f01-8463-ef1768b5f588" containerName="istio-proxy"
Oct 30 22:19:24.621167 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.621153    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4a9121ce-2d96-4f01-8463-ef1768b5f588" containerName="istio-proxy"
Oct 30 22:19:24.621380 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.621214    4871 cpu_manager.go:414] "containerMap: removing container" podUID="78387fab-963c-4f00-9fda-33a5a5ebaee7" containerName="istio-proxy"
Oct 30 22:19:24.621380 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.621222    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="78387fab-963c-4f00-9fda-33a5a5ebaee7" containerName="istio-proxy"
Oct 30 22:19:24.621380 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.621375    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="4a9121ce-2d96-4f01-8463-ef1768b5f588" containerName="istio-proxy"
Oct 30 22:19:24.625077 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.625055    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"
Oct 30 22:19:24.675132 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.675097    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-a-f4a424-openshift-default-dockercfg-rzbmt\""
Oct 30 22:19:24.682875 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.682843    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"]
Oct 30 22:19:24.720454 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.720424    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:24.750613 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.750571    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-istio-data\") pod \"e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5\" (UID: \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\") " pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"
Oct 30 22:19:24.750836 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.750702    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44wqh\" (UniqueName: \"kubernetes.io/projected/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-kube-api-access-44wqh\") pod \"e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5\" (UID: \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\") " pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"
Oct 30 22:19:24.750836 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.750747    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-istio-podinfo\") pod \"e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5\" (UID: \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\") " pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"
Oct 30 22:19:24.750963 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.750868    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-istio-token\") pod \"e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5\" (UID: \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\") " pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"
Oct 30 22:19:24.751022 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.750958    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-istiod-ca-cert\") pod \"e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5\" (UID: \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\") " pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"
Oct 30 22:19:24.751086 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.751022    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-workload-socket\") pod \"e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5\" (UID: \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\") " pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"
Oct 30 22:19:24.751086 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.751067    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-istio-envoy\") pod \"e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5\" (UID: \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\") " pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"
Oct 30 22:19:24.751192 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.751097    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-credential-socket\") pod \"e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5\" (UID: \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\") " pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"
Oct 30 22:19:24.751192 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.751157    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-workload-certs\") pod \"e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5\" (UID: \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\") " pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"
Oct 30 22:19:24.853150 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.853108    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-istio-token\") pod \"e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5\" (UID: \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\") " pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"
Oct 30 22:19:24.853727 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.853183    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-istiod-ca-cert\") pod \"e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5\" (UID: \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\") " pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"
Oct 30 22:19:24.853727 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.853231    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-workload-socket\") pod \"e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5\" (UID: \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\") " pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"
Oct 30 22:19:24.853727 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.853342    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-istio-envoy\") pod \"e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5\" (UID: \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\") " pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"
Oct 30 22:19:24.853727 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.853382    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-credential-socket\") pod \"e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5\" (UID: \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\") " pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"
Oct 30 22:19:24.853727 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.853416    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-workload-certs\") pod \"e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5\" (UID: \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\") " pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"
Oct 30 22:19:24.853727 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.853534    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-istio-data\") pod \"e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5\" (UID: \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\") " pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"
Oct 30 22:19:24.853727 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.853659    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44wqh\" (UniqueName: \"kubernetes.io/projected/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-kube-api-access-44wqh\") pod \"e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5\" (UID: \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\") " pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"
Oct 30 22:19:24.853727 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.853707    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-istio-podinfo\") pod \"e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5\" (UID: \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\") " pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"
Oct 30 22:19:24.855513 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.854638    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-credential-socket\") pod \"e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5\" (UID: \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\") " pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"
Oct 30 22:19:24.855513 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.854653    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-workload-certs\") pod \"e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5\" (UID: \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\") " pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"
Oct 30 22:19:24.855513 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.854892    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-istio-data\") pod \"e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5\" (UID: \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\") " pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"
Oct 30 22:19:24.855513 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.855323    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-istiod-ca-cert\") pod \"e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5\" (UID: \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\") " pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"
Oct 30 22:19:24.855762 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.855589    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-workload-socket\") pod \"e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5\" (UID: \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\") " pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"
Oct 30 22:19:24.879931 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.879863    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-istio-podinfo\") pod \"e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5\" (UID: \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\") " pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"
Oct 30 22:19:24.880728 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.880702    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-istio-envoy\") pod \"e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5\" (UID: \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\") " pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"
Oct 30 22:19:24.905682 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.905643    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:24.913341 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.913271    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-44wqh\" (UniqueName: \"kubernetes.io/projected/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-kube-api-access-44wqh\") pod \"e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5\" (UID: \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\") " pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"
Oct 30 22:19:24.913341 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.913329    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-istio-token\") pod \"e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5\" (UID: \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\") " pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"
Oct 30 22:19:24.954188 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:24.954150    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"
Oct 30 22:19:25.216017 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.211498    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"
Oct 30 22:19:25.361261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.360660    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6674b9d4-017b-4109-9262-bc8ad1244712-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6674b9d4-017b-4109-9262-bc8ad1244712-credential-socket\") pod \"6674b9d4-017b-4109-9262-bc8ad1244712\" (UID: \"6674b9d4-017b-4109-9262-bc8ad1244712\") "
Oct 30 22:19:25.361261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.360738    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6674b9d4-017b-4109-9262-bc8ad1244712-istio-token\" (UniqueName: \"kubernetes.io/projected/6674b9d4-017b-4109-9262-bc8ad1244712-istio-token\") pod \"6674b9d4-017b-4109-9262-bc8ad1244712\" (UID: \"6674b9d4-017b-4109-9262-bc8ad1244712\") "
Oct 30 22:19:25.361261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.360789    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6674b9d4-017b-4109-9262-bc8ad1244712-kube-api-access-66gcx\" (UniqueName: \"kubernetes.io/projected/6674b9d4-017b-4109-9262-bc8ad1244712-kube-api-access-66gcx\") pod \"6674b9d4-017b-4109-9262-bc8ad1244712\" (UID: \"6674b9d4-017b-4109-9262-bc8ad1244712\") "
Oct 30 22:19:25.361261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.360890    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6674b9d4-017b-4109-9262-bc8ad1244712-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6674b9d4-017b-4109-9262-bc8ad1244712-workload-socket\") pod \"6674b9d4-017b-4109-9262-bc8ad1244712\" (UID: \"6674b9d4-017b-4109-9262-bc8ad1244712\") "
Oct 30 22:19:25.361261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.360973    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6674b9d4-017b-4109-9262-bc8ad1244712-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6674b9d4-017b-4109-9262-bc8ad1244712-istio-envoy\") pod \"6674b9d4-017b-4109-9262-bc8ad1244712\" (UID: \"6674b9d4-017b-4109-9262-bc8ad1244712\") "
Oct 30 22:19:25.361261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.361000    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6674b9d4-017b-4109-9262-bc8ad1244712-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6674b9d4-017b-4109-9262-bc8ad1244712-istio-data\") pod \"6674b9d4-017b-4109-9262-bc8ad1244712\" (UID: \"6674b9d4-017b-4109-9262-bc8ad1244712\") "
Oct 30 22:19:25.361261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.361040    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6674b9d4-017b-4109-9262-bc8ad1244712-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6674b9d4-017b-4109-9262-bc8ad1244712-istio-podinfo\") pod \"6674b9d4-017b-4109-9262-bc8ad1244712\" (UID: \"6674b9d4-017b-4109-9262-bc8ad1244712\") "
Oct 30 22:19:25.361261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.361091    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6674b9d4-017b-4109-9262-bc8ad1244712-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6674b9d4-017b-4109-9262-bc8ad1244712-workload-certs\") pod \"6674b9d4-017b-4109-9262-bc8ad1244712\" (UID: \"6674b9d4-017b-4109-9262-bc8ad1244712\") "
Oct 30 22:19:25.361261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.361134    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6674b9d4-017b-4109-9262-bc8ad1244712-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6674b9d4-017b-4109-9262-bc8ad1244712-istiod-ca-cert\") pod \"6674b9d4-017b-4109-9262-bc8ad1244712\" (UID: \"6674b9d4-017b-4109-9262-bc8ad1244712\") "
Oct 30 22:19:25.367187 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.366576    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6674b9d4-017b-4109-9262-bc8ad1244712-credential-socket" pod "6674b9d4-017b-4109-9262-bc8ad1244712" (UID: "6674b9d4-017b-4109-9262-bc8ad1244712"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:19:25.372067 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.369483    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6674b9d4-017b-4109-9262-bc8ad1244712-istiod-ca-cert" pod "6674b9d4-017b-4109-9262-bc8ad1244712" (UID: "6674b9d4-017b-4109-9262-bc8ad1244712"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:19:25.381525 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.381337    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6674b9d4-017b-4109-9262-bc8ad1244712-workload-socket" pod "6674b9d4-017b-4109-9262-bc8ad1244712" (UID: "6674b9d4-017b-4109-9262-bc8ad1244712"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:19:25.381525 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.381434    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6674b9d4-017b-4109-9262-bc8ad1244712-workload-certs" pod "6674b9d4-017b-4109-9262-bc8ad1244712" (UID: "6674b9d4-017b-4109-9262-bc8ad1244712"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:19:25.382018 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.381995    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6674b9d4-017b-4109-9262-bc8ad1244712-istio-data" pod "6674b9d4-017b-4109-9262-bc8ad1244712" (UID: "6674b9d4-017b-4109-9262-bc8ad1244712"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:19:25.387123 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.387059    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6674b9d4-017b-4109-9262-bc8ad1244712-istio-token" pod "6674b9d4-017b-4109-9262-bc8ad1244712" (UID: "6674b9d4-017b-4109-9262-bc8ad1244712"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:19:25.426447 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.425003    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6674b9d4-017b-4109-9262-bc8ad1244712-istio-podinfo" pod "6674b9d4-017b-4109-9262-bc8ad1244712" (UID: "6674b9d4-017b-4109-9262-bc8ad1244712"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:19:25.427107 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.426749    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6674b9d4-017b-4109-9262-bc8ad1244712-istio-envoy" pod "6674b9d4-017b-4109-9262-bc8ad1244712" (UID: "6674b9d4-017b-4109-9262-bc8ad1244712"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:19:25.427107 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.427001    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:25.427107 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.427036    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6674b9d4-017b-4109-9262-bc8ad1244712-kube-api-access-66gcx" pod "6674b9d4-017b-4109-9262-bc8ad1244712" (UID: "6674b9d4-017b-4109-9262-bc8ad1244712"). InnerVolumeSpecName "kube-api-access-66gcx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:19:25.429987 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.429204    4871 generic.go:356] "Generic (PLEG): container finished" podID="6674b9d4-017b-4109-9262-bc8ad1244712" containerID="da178ee02a1013c7b9cfc72cd53ca60dbe18900ff5139ea376e4466cc1b24469" exitCode=0
Oct 30 22:19:25.430381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.430359    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"
Oct 30 22:19:25.432439 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.432308    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856" event={"ID":"6674b9d4-017b-4109-9262-bc8ad1244712","Type":"ContainerDied","Data":"da178ee02a1013c7b9cfc72cd53ca60dbe18900ff5139ea376e4466cc1b24469"}
Oct 30 22:19:25.432439 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.432395    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856" event={"ID":"6674b9d4-017b-4109-9262-bc8ad1244712","Type":"ContainerDied","Data":"ba9c1c10322b8f7bc663355853ecf0824d275a080ef2d0700a10e67859bb47f5"}
Oct 30 22:19:25.432439 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.432414    4871 scope.go:122] "RemoveContainer" containerID="da178ee02a1013c7b9cfc72cd53ca60dbe18900ff5139ea376e4466cc1b24469"
Oct 30 22:19:25.449813 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.449745    4871 scope.go:122] "RemoveContainer" containerID="7b93e238cb14ccef31ed03ca1a25e2ede53c102435d6fb5fe8b45a2c87af535b"
Oct 30 22:19:25.461055 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.461020    4871 scope.go:122] "RemoveContainer" containerID="da178ee02a1013c7b9cfc72cd53ca60dbe18900ff5139ea376e4466cc1b24469"
Oct 30 22:19:25.461518 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:19:25.461489    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da178ee02a1013c7b9cfc72cd53ca60dbe18900ff5139ea376e4466cc1b24469\": container with ID starting with da178ee02a1013c7b9cfc72cd53ca60dbe18900ff5139ea376e4466cc1b24469 not found: ID does not exist" containerID="da178ee02a1013c7b9cfc72cd53ca60dbe18900ff5139ea376e4466cc1b24469"
Oct 30 22:19:25.461633 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.461524    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da178ee02a1013c7b9cfc72cd53ca60dbe18900ff5139ea376e4466cc1b24469"} err="failed to get container status \"da178ee02a1013c7b9cfc72cd53ca60dbe18900ff5139ea376e4466cc1b24469\": rpc error: code = NotFound desc = could not find container \"da178ee02a1013c7b9cfc72cd53ca60dbe18900ff5139ea376e4466cc1b24469\": container with ID starting with da178ee02a1013c7b9cfc72cd53ca60dbe18900ff5139ea376e4466cc1b24469 not found: ID does not exist"
Oct 30 22:19:25.461633 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.461552    4871 scope.go:122] "RemoveContainer" containerID="7b93e238cb14ccef31ed03ca1a25e2ede53c102435d6fb5fe8b45a2c87af535b"
Oct 30 22:19:25.461915 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:19:25.461879    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b93e238cb14ccef31ed03ca1a25e2ede53c102435d6fb5fe8b45a2c87af535b\": container with ID starting with 7b93e238cb14ccef31ed03ca1a25e2ede53c102435d6fb5fe8b45a2c87af535b not found: ID does not exist" containerID="7b93e238cb14ccef31ed03ca1a25e2ede53c102435d6fb5fe8b45a2c87af535b"
Oct 30 22:19:25.462007 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.461928    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b93e238cb14ccef31ed03ca1a25e2ede53c102435d6fb5fe8b45a2c87af535b"} err="failed to get container status \"7b93e238cb14ccef31ed03ca1a25e2ede53c102435d6fb5fe8b45a2c87af535b\": rpc error: code = NotFound desc = could not find container \"7b93e238cb14ccef31ed03ca1a25e2ede53c102435d6fb5fe8b45a2c87af535b\": container with ID starting with 7b93e238cb14ccef31ed03ca1a25e2ede53c102435d6fb5fe8b45a2c87af535b not found: ID does not exist"
Oct 30 22:19:25.462827 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.462806    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6674b9d4-017b-4109-9262-bc8ad1244712-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:25.462948 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.462928    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6674b9d4-017b-4109-9262-bc8ad1244712-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:25.462948 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.462945    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6674b9d4-017b-4109-9262-bc8ad1244712-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:25.463115 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.462964    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6674b9d4-017b-4109-9262-bc8ad1244712-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:25.463115 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.463030    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6674b9d4-017b-4109-9262-bc8ad1244712-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:25.463115 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.463041    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6674b9d4-017b-4109-9262-bc8ad1244712-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:25.463115 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.463054    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6674b9d4-017b-4109-9262-bc8ad1244712-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:25.463115 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.463064    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66gcx\" (UniqueName: \"kubernetes.io/projected/6674b9d4-017b-4109-9262-bc8ad1244712-kube-api-access-66gcx\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:25.463115 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.463073    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6674b9d4-017b-4109-9262-bc8ad1244712-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:25.515385 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.513052    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"]
Oct 30 22:19:25.518316 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.518275    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:19:25.518574 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.518552    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:19:25.518723 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.518705    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:19:25.617134 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.617066    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:25.618443 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.618377    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"]
Oct 30 22:19:25.633138 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.630939    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-c4bfac-openshift-default-67fb85b5bc-tf856"]
Oct 30 22:19:25.963558 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:25.963478    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6674b9d4-017b-4109-9262-bc8ad1244712" path="/var/lib/kubelet/pods/6674b9d4-017b-4109-9262-bc8ad1244712/volumes"
Oct 30 22:19:26.439462 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:26.439420    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5" event={"ID":"d6407e4e-8850-4c67-8ad5-93c7fde3ea55","Type":"ContainerStarted","Data":"ff5ec537f48ff74c1c24133feccb802e9c75d66e1b69adb617f0d0a9543743c3"}
Oct 30 22:19:26.439676 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:26.439491    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5" event={"ID":"d6407e4e-8850-4c67-8ad5-93c7fde3ea55","Type":"ContainerStarted","Data":"b3e1d0c483d523cf1e8e923539a99fea89bd8390ac8f6ad596cca21976f34829"}
Oct 30 22:19:26.560063 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:26.559998    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5" podStartSLOduration=2.559982702 podStartE2EDuration="2.559982702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:19:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:19:26.557402861 +0000 UTC m=+2508.734891480" watchObservedRunningTime="2026-10-30 22:19:26.559982702 +0000 UTC m=+2508.737471319"
Oct 30 22:19:26.873595 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:26.873553    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:26.955330 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:26.955287    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"
Oct 30 22:19:26.960913 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:26.960883    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"
Oct 30 22:19:27.023169 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:27.023114    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:27.196145 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:27.196054    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:27.376220 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:27.376167    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:27.450395 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:27.450302    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"
Oct 30 22:19:27.451381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:27.451359    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"
Oct 30 22:19:27.875143 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:27.875109    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:28.468522 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:28.468482    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-63a282-5978dfffdf-wtlnf" podUID="dd380e6f-9f62-45c9-ba8c-a4b16e749436" containerName="payload-processing" containerID="cri-o://1c136abc1d059a566e10da3a2bdb4aa05393f86ffdc7d2a145458fafb891f70d" gracePeriod=25
Oct 30 22:19:28.571601 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:28.571527    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-63a282-6c49469979-hf22h" podUID="8c399f24-ffaf-4649-89c9-49de27e94c41" containerName="payload-pre-processing" containerID="cri-o://af40da7fc0e7cab19c905ede21a0b0036b271d03d51cf74197e840e2e37f06e0" gracePeriod=25
Oct 30 22:19:28.757053 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:28.757020    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-63a282-5978dfffdf-wtlnf"
Oct 30 22:19:28.874474 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:28.874446    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-63a282-6c49469979-hf22h"
Oct 30 22:19:28.924413 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:28.924375    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dd380e6f-9f62-45c9-ba8c-a4b16e749436-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd380e6f-9f62-45c9-ba8c-a4b16e749436-plugins-config-volume\") pod \"dd380e6f-9f62-45c9-ba8c-a4b16e749436\" (UID: \"dd380e6f-9f62-45c9-ba8c-a4b16e749436\") "
Oct 30 22:19:28.924623 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:28.924606    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dd380e6f-9f62-45c9-ba8c-a4b16e749436-kube-api-access-2p6ww\" (UniqueName: \"kubernetes.io/projected/dd380e6f-9f62-45c9-ba8c-a4b16e749436-kube-api-access-2p6ww\") pod \"dd380e6f-9f62-45c9-ba8c-a4b16e749436\" (UID: \"dd380e6f-9f62-45c9-ba8c-a4b16e749436\") "
Oct 30 22:19:28.924785 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:28.924756    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd380e6f-9f62-45c9-ba8c-a4b16e749436-plugins-config-volume" pod "dd380e6f-9f62-45c9-ba8c-a4b16e749436" (UID: "dd380e6f-9f62-45c9-ba8c-a4b16e749436"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:19:28.925023 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:28.925005    4871 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd380e6f-9f62-45c9-ba8c-a4b16e749436-plugins-config-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:28.926710 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:28.926685    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd380e6f-9f62-45c9-ba8c-a4b16e749436-kube-api-access-2p6ww" pod "dd380e6f-9f62-45c9-ba8c-a4b16e749436" (UID: "dd380e6f-9f62-45c9-ba8c-a4b16e749436"). InnerVolumeSpecName "kube-api-access-2p6ww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:19:29.026517 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:29.026424    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8c399f24-ffaf-4649-89c9-49de27e94c41-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c399f24-ffaf-4649-89c9-49de27e94c41-plugins-config-volume\") pod \"8c399f24-ffaf-4649-89c9-49de27e94c41\" (UID: \"8c399f24-ffaf-4649-89c9-49de27e94c41\") "
Oct 30 22:19:29.026517 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:29.026498    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8c399f24-ffaf-4649-89c9-49de27e94c41-kube-api-access-k7hbm\" (UniqueName: \"kubernetes.io/projected/8c399f24-ffaf-4649-89c9-49de27e94c41-kube-api-access-k7hbm\") pod \"8c399f24-ffaf-4649-89c9-49de27e94c41\" (UID: \"8c399f24-ffaf-4649-89c9-49de27e94c41\") "
Oct 30 22:19:29.026959 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:29.026931    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c399f24-ffaf-4649-89c9-49de27e94c41-plugins-config-volume" pod "8c399f24-ffaf-4649-89c9-49de27e94c41" (UID: "8c399f24-ffaf-4649-89c9-49de27e94c41"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:19:29.027043 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:29.027023    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2p6ww\" (UniqueName: \"kubernetes.io/projected/dd380e6f-9f62-45c9-ba8c-a4b16e749436-kube-api-access-2p6ww\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:29.027101 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:29.027049    4871 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c399f24-ffaf-4649-89c9-49de27e94c41-plugins-config-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:29.028749 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:29.028727    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c399f24-ffaf-4649-89c9-49de27e94c41-kube-api-access-k7hbm" pod "8c399f24-ffaf-4649-89c9-49de27e94c41" (UID: "8c399f24-ffaf-4649-89c9-49de27e94c41"). InnerVolumeSpecName "kube-api-access-k7hbm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:19:29.128944 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:29.128902    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7hbm\" (UniqueName: \"kubernetes.io/projected/8c399f24-ffaf-4649-89c9-49de27e94c41-kube-api-access-k7hbm\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:29.213059 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:29.213006    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:29.465926 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:29.465887    4871 generic.go:356] "Generic (PLEG): container finished" podID="dd380e6f-9f62-45c9-ba8c-a4b16e749436" containerID="1c136abc1d059a566e10da3a2bdb4aa05393f86ffdc7d2a145458fafb891f70d" exitCode=0
Oct 30 22:19:29.466136 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:29.465944    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-63a282-5978dfffdf-wtlnf"
Oct 30 22:19:29.466136 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:29.465972    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-63a282-5978dfffdf-wtlnf" event={"ID":"dd380e6f-9f62-45c9-ba8c-a4b16e749436","Type":"ContainerDied","Data":"1c136abc1d059a566e10da3a2bdb4aa05393f86ffdc7d2a145458fafb891f70d"}
Oct 30 22:19:29.466136 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:29.466020    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-63a282-5978dfffdf-wtlnf" event={"ID":"dd380e6f-9f62-45c9-ba8c-a4b16e749436","Type":"ContainerDied","Data":"96f1865ded9fa5d91c97260b91d7a34c37579f7ae4b016c87ee219e78ed8b56c"}
Oct 30 22:19:29.466136 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:29.466041    4871 scope.go:122] "RemoveContainer" containerID="1c136abc1d059a566e10da3a2bdb4aa05393f86ffdc7d2a145458fafb891f70d"
Oct 30 22:19:29.467550 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:29.467528    4871 generic.go:356] "Generic (PLEG): container finished" podID="8c399f24-ffaf-4649-89c9-49de27e94c41" containerID="af40da7fc0e7cab19c905ede21a0b0036b271d03d51cf74197e840e2e37f06e0" exitCode=0
Oct 30 22:19:29.467659 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:29.467572    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-63a282-6c49469979-hf22h"
Oct 30 22:19:29.467659 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:29.467606    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-63a282-6c49469979-hf22h" event={"ID":"8c399f24-ffaf-4649-89c9-49de27e94c41","Type":"ContainerDied","Data":"af40da7fc0e7cab19c905ede21a0b0036b271d03d51cf74197e840e2e37f06e0"}
Oct 30 22:19:29.467659 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:29.467650    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-63a282-6c49469979-hf22h" event={"ID":"8c399f24-ffaf-4649-89c9-49de27e94c41","Type":"ContainerDied","Data":"ae9c956f5f167a4eb409e116d807e87375aa917afba57dec2020c46bcb71e2fc"}
Oct 30 22:19:29.481280 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:29.481025    4871 scope.go:122] "RemoveContainer" containerID="1c136abc1d059a566e10da3a2bdb4aa05393f86ffdc7d2a145458fafb891f70d"
Oct 30 22:19:29.481638 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:19:29.481393    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c136abc1d059a566e10da3a2bdb4aa05393f86ffdc7d2a145458fafb891f70d\": container with ID starting with 1c136abc1d059a566e10da3a2bdb4aa05393f86ffdc7d2a145458fafb891f70d not found: ID does not exist" containerID="1c136abc1d059a566e10da3a2bdb4aa05393f86ffdc7d2a145458fafb891f70d"
Oct 30 22:19:29.481638 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:29.481429    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c136abc1d059a566e10da3a2bdb4aa05393f86ffdc7d2a145458fafb891f70d"} err="failed to get container status \"1c136abc1d059a566e10da3a2bdb4aa05393f86ffdc7d2a145458fafb891f70d\": rpc error: code = NotFound desc = could not find container \"1c136abc1d059a566e10da3a2bdb4aa05393f86ffdc7d2a145458fafb891f70d\": container with ID starting with 1c136abc1d059a566e10da3a2bdb4aa05393f86ffdc7d2a145458fafb891f70d not found: ID does not exist"
Oct 30 22:19:29.481638 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:29.481446    4871 scope.go:122] "RemoveContainer" containerID="af40da7fc0e7cab19c905ede21a0b0036b271d03d51cf74197e840e2e37f06e0"
Oct 30 22:19:29.494864 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:29.494839    4871 scope.go:122] "RemoveContainer" containerID="af40da7fc0e7cab19c905ede21a0b0036b271d03d51cf74197e840e2e37f06e0"
Oct 30 22:19:29.495167 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:19:29.495146    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af40da7fc0e7cab19c905ede21a0b0036b271d03d51cf74197e840e2e37f06e0\": container with ID starting with af40da7fc0e7cab19c905ede21a0b0036b271d03d51cf74197e840e2e37f06e0 not found: ID does not exist" containerID="af40da7fc0e7cab19c905ede21a0b0036b271d03d51cf74197e840e2e37f06e0"
Oct 30 22:19:29.495260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:29.495175    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af40da7fc0e7cab19c905ede21a0b0036b271d03d51cf74197e840e2e37f06e0"} err="failed to get container status \"af40da7fc0e7cab19c905ede21a0b0036b271d03d51cf74197e840e2e37f06e0\": rpc error: code = NotFound desc = could not find container \"af40da7fc0e7cab19c905ede21a0b0036b271d03d51cf74197e840e2e37f06e0\": container with ID starting with af40da7fc0e7cab19c905ede21a0b0036b271d03d51cf74197e840e2e37f06e0 not found: ID does not exist"
Oct 30 22:19:29.625015 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:29.624978    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-63a282-6c49469979-hf22h"]
Oct 30 22:19:29.671708 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:29.671675    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-63a282-6c49469979-hf22h"]
Oct 30 22:19:29.735540 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:29.735442    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-63a282-5978dfffdf-wtlnf"]
Oct 30 22:19:29.741037 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:29.741004    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-63a282-5978dfffdf-wtlnf"]
Oct 30 22:19:29.968285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:29.968217    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8c399f24-ffaf-4649-89c9-49de27e94c41" path="/var/lib/kubelet/pods/8c399f24-ffaf-4649-89c9-49de27e94c41/volumes"
Oct 30 22:19:29.968688 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:29.968674    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dd380e6f-9f62-45c9-ba8c-a4b16e749436" path="/var/lib/kubelet/pods/dd380e6f-9f62-45c9-ba8c-a4b16e749436/volumes"
Oct 30 22:19:30.358507 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:30.358473    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:30.517313 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:30.511550    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:30.720554 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:30.720518    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:31.008465 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:31.006314    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:31.159514 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:31.159483    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:31.358515 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:31.358405    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:31.520337 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:31.520297    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:34.246382 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:34.246341    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:35.470791 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:35.470747    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:35.623270 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:35.623195    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:35.627778 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:19:35.627741    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:19:37.975860 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:37.975827    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:38.156265 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:38.156186    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:38.394515 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:38.394475    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p"]
Oct 30 22:19:38.394762 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:38.394732    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p" podUID="9a6d90d2-019e-41e6-9d6f-a64f537f4963" containerName="maas-api" containerID="cri-o://86315d3a75d46e072124bc72d96c465602d8f18dd00bd7e5c9a0aef6e26bffbf" gracePeriod=30
Oct 30 22:19:38.501033 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:38.500998    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:38.600820 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:38.600786    4871 generic.go:356] "Generic (PLEG): container finished" podID="9a6d90d2-019e-41e6-9d6f-a64f537f4963" containerID="86315d3a75d46e072124bc72d96c465602d8f18dd00bd7e5c9a0aef6e26bffbf" exitCode=0
Oct 30 22:19:38.601024 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:38.600867    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p" event={"ID":"9a6d90d2-019e-41e6-9d6f-a64f537f4963","Type":"ContainerDied","Data":"86315d3a75d46e072124bc72d96c465602d8f18dd00bd7e5c9a0aef6e26bffbf"}
Oct 30 22:19:38.739970 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:38.739943    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p"
Oct 30 22:19:38.840233 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:38.840194    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-fe745d-85cf688ff4-xh6cf"]
Oct 30 22:19:38.840482 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:38.840454    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-fe745d-85cf688ff4-xh6cf" podUID="baab5eee-6bad-4516-9084-049003c4f2d3" containerName="payload-processing" containerID="cri-o://11e191ed2c0e8f74506405dd4e687b4db3b7649bcdd948dad2b078e71041e59a" gracePeriod=30
Oct 30 22:19:38.864319 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:38.864289    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9a6d90d2-019e-41e6-9d6f-a64f537f4963-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a6d90d2-019e-41e6-9d6f-a64f537f4963-openshift-service-ca\") pod \"9a6d90d2-019e-41e6-9d6f-a64f537f4963\" (UID: \"9a6d90d2-019e-41e6-9d6f-a64f537f4963\") "
Oct 30 22:19:38.864485 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:38.864392    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9a6d90d2-019e-41e6-9d6f-a64f537f4963-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a6d90d2-019e-41e6-9d6f-a64f537f4963-maas-api-tls\") pod \"9a6d90d2-019e-41e6-9d6f-a64f537f4963\" (UID: \"9a6d90d2-019e-41e6-9d6f-a64f537f4963\") "
Oct 30 22:19:38.864485 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:38.864473    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9a6d90d2-019e-41e6-9d6f-a64f537f4963-kube-api-access-wlb2n\" (UniqueName: \"kubernetes.io/projected/9a6d90d2-019e-41e6-9d6f-a64f537f4963-kube-api-access-wlb2n\") pod \"9a6d90d2-019e-41e6-9d6f-a64f537f4963\" (UID: \"9a6d90d2-019e-41e6-9d6f-a64f537f4963\") "
Oct 30 22:19:38.864619 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:38.864525    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9a6d90d2-019e-41e6-9d6f-a64f537f4963-metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a6d90d2-019e-41e6-9d6f-a64f537f4963-metrics-certs\") pod \"9a6d90d2-019e-41e6-9d6f-a64f537f4963\" (UID: \"9a6d90d2-019e-41e6-9d6f-a64f537f4963\") "
Oct 30 22:19:38.864726 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:38.864705    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a6d90d2-019e-41e6-9d6f-a64f537f4963-openshift-service-ca" pod "9a6d90d2-019e-41e6-9d6f-a64f537f4963" (UID: "9a6d90d2-019e-41e6-9d6f-a64f537f4963"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:19:38.867146 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:38.867102    4871 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a6d90d2-019e-41e6-9d6f-a64f537f4963-openshift-service-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:38.867560 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:38.867529    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a6d90d2-019e-41e6-9d6f-a64f537f4963-kube-api-access-wlb2n" pod "9a6d90d2-019e-41e6-9d6f-a64f537f4963" (UID: "9a6d90d2-019e-41e6-9d6f-a64f537f4963"). InnerVolumeSpecName "kube-api-access-wlb2n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:19:38.867682 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:38.867662    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a6d90d2-019e-41e6-9d6f-a64f537f4963-maas-api-tls" pod "9a6d90d2-019e-41e6-9d6f-a64f537f4963" (UID: "9a6d90d2-019e-41e6-9d6f-a64f537f4963"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:19:38.867757 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:38.867677    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a6d90d2-019e-41e6-9d6f-a64f537f4963-metrics-certs" pod "9a6d90d2-019e-41e6-9d6f-a64f537f4963" (UID: "9a6d90d2-019e-41e6-9d6f-a64f537f4963"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:19:38.967928 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:38.967893    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a6d90d2-019e-41e6-9d6f-a64f537f4963-maas-api-tls\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:38.967928 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:38.967929    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wlb2n\" (UniqueName: \"kubernetes.io/projected/9a6d90d2-019e-41e6-9d6f-a64f537f4963-kube-api-access-wlb2n\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:38.968150 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:38.967946    4871 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a6d90d2-019e-41e6-9d6f-a64f537f4963-metrics-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:39.001848 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.001797    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:39.112024 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.111997    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-fe745d-85cf688ff4-xh6cf"
Oct 30 22:19:39.158287 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.158206    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:39.271506 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.271422    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/baab5eee-6bad-4516-9084-049003c4f2d3-kube-api-access-4rvwt\" (UniqueName: \"kubernetes.io/projected/baab5eee-6bad-4516-9084-049003c4f2d3-kube-api-access-4rvwt\") pod \"baab5eee-6bad-4516-9084-049003c4f2d3\" (UID: \"baab5eee-6bad-4516-9084-049003c4f2d3\") "
Oct 30 22:19:39.271506 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.271466    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/baab5eee-6bad-4516-9084-049003c4f2d3-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/baab5eee-6bad-4516-9084-049003c4f2d3-plugins-config-volume\") pod \"baab5eee-6bad-4516-9084-049003c4f2d3\" (UID: \"baab5eee-6bad-4516-9084-049003c4f2d3\") "
Oct 30 22:19:39.271883 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.271860    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/baab5eee-6bad-4516-9084-049003c4f2d3-plugins-config-volume" pod "baab5eee-6bad-4516-9084-049003c4f2d3" (UID: "baab5eee-6bad-4516-9084-049003c4f2d3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:19:39.273662 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.273639    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/baab5eee-6bad-4516-9084-049003c4f2d3-kube-api-access-4rvwt" pod "baab5eee-6bad-4516-9084-049003c4f2d3" (UID: "baab5eee-6bad-4516-9084-049003c4f2d3"). InnerVolumeSpecName "kube-api-access-4rvwt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:19:39.374252 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.374195    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rvwt\" (UniqueName: \"kubernetes.io/projected/baab5eee-6bad-4516-9084-049003c4f2d3-kube-api-access-4rvwt\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:39.374494 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.374474    4871 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/baab5eee-6bad-4516-9084-049003c4f2d3-plugins-config-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:39.610641 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.610552    4871 generic.go:356] "Generic (PLEG): container finished" podID="baab5eee-6bad-4516-9084-049003c4f2d3" containerID="11e191ed2c0e8f74506405dd4e687b4db3b7649bcdd948dad2b078e71041e59a" exitCode=0
Oct 30 22:19:39.610641 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.610604    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-fe745d-85cf688ff4-xh6cf"
Oct 30 22:19:39.610854 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.610633    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-fe745d-85cf688ff4-xh6cf" event={"ID":"baab5eee-6bad-4516-9084-049003c4f2d3","Type":"ContainerDied","Data":"11e191ed2c0e8f74506405dd4e687b4db3b7649bcdd948dad2b078e71041e59a"}
Oct 30 22:19:39.610854 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.610691    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-fe745d-85cf688ff4-xh6cf" event={"ID":"baab5eee-6bad-4516-9084-049003c4f2d3","Type":"ContainerDied","Data":"40cf307e8869fa18075084c7d594b8680707bb6580fadb8eff1923e38dd9d4c2"}
Oct 30 22:19:39.610854 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.610717    4871 scope.go:122] "RemoveContainer" containerID="11e191ed2c0e8f74506405dd4e687b4db3b7649bcdd948dad2b078e71041e59a"
Oct 30 22:19:39.612545 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.612518    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p"
Oct 30 22:19:39.612627 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.612517    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p" event={"ID":"9a6d90d2-019e-41e6-9d6f-a64f537f4963","Type":"ContainerDied","Data":"f8257b58ef2168158b55ceff0fe2aeed2d3fbe027f138bb1fc85eb25f6bb219e"}
Oct 30 22:19:39.623423 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.623270    4871 scope.go:122] "RemoveContainer" containerID="11e191ed2c0e8f74506405dd4e687b4db3b7649bcdd948dad2b078e71041e59a"
Oct 30 22:19:39.623685 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:19:39.623665    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11e191ed2c0e8f74506405dd4e687b4db3b7649bcdd948dad2b078e71041e59a\": container with ID starting with 11e191ed2c0e8f74506405dd4e687b4db3b7649bcdd948dad2b078e71041e59a not found: ID does not exist" containerID="11e191ed2c0e8f74506405dd4e687b4db3b7649bcdd948dad2b078e71041e59a"
Oct 30 22:19:39.623754 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.623704    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11e191ed2c0e8f74506405dd4e687b4db3b7649bcdd948dad2b078e71041e59a"} err="failed to get container status \"11e191ed2c0e8f74506405dd4e687b4db3b7649bcdd948dad2b078e71041e59a\": rpc error: code = NotFound desc = could not find container \"11e191ed2c0e8f74506405dd4e687b4db3b7649bcdd948dad2b078e71041e59a\": container with ID starting with 11e191ed2c0e8f74506405dd4e687b4db3b7649bcdd948dad2b078e71041e59a not found: ID does not exist"
Oct 30 22:19:39.623754 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.623734    4871 scope.go:122] "RemoveContainer" containerID="86315d3a75d46e072124bc72d96c465602d8f18dd00bd7e5c9a0aef6e26bffbf"
Oct 30 22:19:39.773351 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.773315    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-fe745d-85cf688ff4-xh6cf"]
Oct 30 22:19:39.820835 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.820795    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-fe745d-85cf688ff4-xh6cf"]
Oct 30 22:19:39.881140 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.881097    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p"]
Oct 30 22:19:39.887832 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.887799    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-fe745d-54fbbf5584-c8x5p"]
Oct 30 22:19:39.967631 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.967593    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9a6d90d2-019e-41e6-9d6f-a64f537f4963" path="/var/lib/kubelet/pods/9a6d90d2-019e-41e6-9d6f-a64f537f4963/volumes"
Oct 30 22:19:39.968071 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.968056    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="baab5eee-6bad-4516-9084-049003c4f2d3" path="/var/lib/kubelet/pods/baab5eee-6bad-4516-9084-049003c4f2d3/volumes"
Oct 30 22:19:39.987861 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.987819    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx"]
Oct 30 22:19:39.988440 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.988424    4871 cpu_manager.go:414] "containerMap: removing container" podUID="6674b9d4-017b-4109-9262-bc8ad1244712" containerName="istio-proxy"
Oct 30 22:19:39.988495 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.988442    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6674b9d4-017b-4109-9262-bc8ad1244712" containerName="istio-proxy"
Oct 30 22:19:39.988495 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.988453    4871 cpu_manager.go:414] "containerMap: removing container" podUID="8c399f24-ffaf-4649-89c9-49de27e94c41" containerName="payload-pre-processing"
Oct 30 22:19:39.988495 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.988458    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8c399f24-ffaf-4649-89c9-49de27e94c41" containerName="payload-pre-processing"
Oct 30 22:19:39.988495 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.988478    4871 cpu_manager.go:414] "containerMap: removing container" podUID="9a6d90d2-019e-41e6-9d6f-a64f537f4963" containerName="maas-api"
Oct 30 22:19:39.988495 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.988482    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9a6d90d2-019e-41e6-9d6f-a64f537f4963" containerName="maas-api"
Oct 30 22:19:39.988495 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.988490    4871 cpu_manager.go:414] "containerMap: removing container" podUID="6674b9d4-017b-4109-9262-bc8ad1244712" containerName="istio-proxy"
Oct 30 22:19:39.988495 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.988494    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6674b9d4-017b-4109-9262-bc8ad1244712" containerName="istio-proxy"
Oct 30 22:19:39.988721 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.988519    4871 cpu_manager.go:414] "containerMap: removing container" podUID="dd380e6f-9f62-45c9-ba8c-a4b16e749436" containerName="payload-processing"
Oct 30 22:19:39.988721 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.988524    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dd380e6f-9f62-45c9-ba8c-a4b16e749436" containerName="payload-processing"
Oct 30 22:19:39.988721 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.988539    4871 cpu_manager.go:414] "containerMap: removing container" podUID="baab5eee-6bad-4516-9084-049003c4f2d3" containerName="payload-processing"
Oct 30 22:19:39.988721 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.988544    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="baab5eee-6bad-4516-9084-049003c4f2d3" containerName="payload-processing"
Oct 30 22:19:39.988721 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.988619    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="6674b9d4-017b-4109-9262-bc8ad1244712" containerName="istio-proxy"
Oct 30 22:19:39.988721 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.988626    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="8c399f24-ffaf-4649-89c9-49de27e94c41" containerName="payload-pre-processing"
Oct 30 22:19:39.988721 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.988637    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="9a6d90d2-019e-41e6-9d6f-a64f537f4963" containerName="maas-api"
Oct 30 22:19:39.988721 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.988647    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="dd380e6f-9f62-45c9-ba8c-a4b16e749436" containerName="payload-processing"
Oct 30 22:19:39.988721 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.988657    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="baab5eee-6bad-4516-9084-049003c4f2d3" containerName="payload-processing"
Oct 30 22:19:39.993462 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:39.993437    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx"
Oct 30 22:19:40.039889 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:40.039853    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:40.043540 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:40.043511    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-a-3cecc7-serving-cert\""
Oct 30 22:19:40.048541 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:40.048512    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx"]
Oct 30 22:19:40.082033 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:40.081988    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rqhh\" (UniqueName: \"kubernetes.io/projected/53120b48-0737-4956-965b-bf544e8c1096-kube-api-access-4rqhh\") pod \"maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx\" (UID: \"53120b48-0737-4956-965b-bf544e8c1096\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx"
Oct 30 22:19:40.082222 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:40.082044    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53120b48-0737-4956-965b-bf544e8c1096-maas-api-tls\") pod \"maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx\" (UID: \"53120b48-0737-4956-965b-bf544e8c1096\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx"
Oct 30 22:19:40.082368 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:40.082265    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53120b48-0737-4956-965b-bf544e8c1096-metrics-certs\") pod \"maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx\" (UID: \"53120b48-0737-4956-965b-bf544e8c1096\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx"
Oct 30 22:19:40.082368 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:40.082324    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53120b48-0737-4956-965b-bf544e8c1096-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx\" (UID: \"53120b48-0737-4956-965b-bf544e8c1096\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx"
Oct 30 22:19:40.183190 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:40.183098    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53120b48-0737-4956-965b-bf544e8c1096-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx\" (UID: \"53120b48-0737-4956-965b-bf544e8c1096\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx"
Oct 30 22:19:40.183363 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:40.183196    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rqhh\" (UniqueName: \"kubernetes.io/projected/53120b48-0737-4956-965b-bf544e8c1096-kube-api-access-4rqhh\") pod \"maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx\" (UID: \"53120b48-0737-4956-965b-bf544e8c1096\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx"
Oct 30 22:19:40.183363 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:40.183228    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53120b48-0737-4956-965b-bf544e8c1096-maas-api-tls\") pod \"maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx\" (UID: \"53120b48-0737-4956-965b-bf544e8c1096\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx"
Oct 30 22:19:40.183363 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:40.183350    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53120b48-0737-4956-965b-bf544e8c1096-metrics-certs\") pod \"maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx\" (UID: \"53120b48-0737-4956-965b-bf544e8c1096\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx"
Oct 30 22:19:40.183507 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:19:40.183428    4871 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-praxis-a-3cecc7-serving-cert: secret "maas-api-e2e-praxis-a-3cecc7-serving-cert" not found
Oct 30 22:19:40.183554 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:19:40.183507    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53120b48-0737-4956-965b-bf544e8c1096-maas-api-tls podName:53120b48-0737-4956-965b-bf544e8c1096 nodeName:}" failed. No retries permitted until 2026-10-30 22:19:40.683486916 +0000 UTC m=+2522.860975518 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/53120b48-0737-4956-965b-bf544e8c1096-maas-api-tls") pod "maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx" (UID: "53120b48-0737-4956-965b-bf544e8c1096") : secret "maas-api-e2e-praxis-a-3cecc7-serving-cert" not found
Oct 30 22:19:40.183841 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:40.183820    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53120b48-0737-4956-965b-bf544e8c1096-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx\" (UID: \"53120b48-0737-4956-965b-bf544e8c1096\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx"
Oct 30 22:19:40.186044 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:40.186018    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53120b48-0737-4956-965b-bf544e8c1096-metrics-certs\") pod \"maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx\" (UID: \"53120b48-0737-4956-965b-bf544e8c1096\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx"
Oct 30 22:19:40.193024 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:40.192992    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:40.261262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:40.260507    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rqhh\" (UniqueName: \"kubernetes.io/projected/53120b48-0737-4956-965b-bf544e8c1096-kube-api-access-4rqhh\") pod \"maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx\" (UID: \"53120b48-0737-4956-965b-bf544e8c1096\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx"
Oct 30 22:19:40.347766 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:40.347728    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:40.501555 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:40.501470    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:40.688951 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:40.688909    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53120b48-0737-4956-965b-bf544e8c1096-maas-api-tls\") pod \"maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx\" (UID: \"53120b48-0737-4956-965b-bf544e8c1096\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx"
Oct 30 22:19:40.691520 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:40.691498    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53120b48-0737-4956-965b-bf544e8c1096-maas-api-tls\") pod \"maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx\" (UID: \"53120b48-0737-4956-965b-bf544e8c1096\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx"
Oct 30 22:19:40.907700 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:40.907659    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx"
Oct 30 22:19:40.985758 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:40.985699    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:41.288797 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:41.288766    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx"]
Oct 30 22:19:41.290265 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:19:41.290220    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53120b48_0737_4956_965b_bf544e8c1096.slice/crio-fbe0d7bda18ac5a2f375a6be0be2eb5d874b8195575b69d2a39793cbe3d9f516 WatchSource:0}: Error finding container fbe0d7bda18ac5a2f375a6be0be2eb5d874b8195575b69d2a39793cbe3d9f516: Status 404 returned error can't find the container with id fbe0d7bda18ac5a2f375a6be0be2eb5d874b8195575b69d2a39793cbe3d9f516
Oct 30 22:19:41.629693 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:41.629657    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx" event={"ID":"53120b48-0737-4956-965b-bf544e8c1096","Type":"ContainerStarted","Data":"fbe0d7bda18ac5a2f375a6be0be2eb5d874b8195575b69d2a39793cbe3d9f516"}
Oct 30 22:19:42.637857 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:42.637814    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx" event={"ID":"53120b48-0737-4956-965b-bf544e8c1096","Type":"ContainerStarted","Data":"0c1800f416abdf4483d07aafdf87668edee24edcf7f9a0c0dbf87d43ea510bf7"}
Oct 30 22:19:42.638286 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:42.637888    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx"
Oct 30 22:19:42.751165 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:42.751102    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx" podStartSLOduration=2.717434738 podStartE2EDuration="3.751086571s" totalImagesPullingTime="1.033651833s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:19:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:19:42.749517036 +0000 UTC m=+2524.927005653" watchObservedRunningTime="2026-10-30 22:19:42.751086571 +0000 UTC m=+2524.928575187"
Oct 30 22:19:42.985418 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:42.985333    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:45.284088 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:45.284053    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:46.911565 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:46.911515    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:47.065272 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:47.065221    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:47.282639 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:47.282532    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:48.651839 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:48.651804    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx"
Oct 30 22:19:49.950101 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:49.950060    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:50.100307 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:50.100234    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:50.541851 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:50.541809    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"]
Oct 30 22:19:50.542533 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:50.542514    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="6674b9d4-017b-4109-9262-bc8ad1244712" containerName="istio-proxy"
Oct 30 22:19:50.546334 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:50.546312    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"
Oct 30 22:19:50.604646 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:50.604613    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-a-f7c488-openshift-default-dockercfg-2kvn7\""
Oct 30 22:19:50.692345 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:50.692298    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"]
Oct 30 22:19:50.699147 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:50.695050    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:50.746271 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:50.734895    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d075c21-db31-400b-b173-02391280ad9f-workload-certs\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8\" (UID: \"1d075c21-db31-400b-b173-02391280ad9f\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"
Oct 30 22:19:50.746734 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:50.746696    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d075c21-db31-400b-b173-02391280ad9f-credential-socket\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8\" (UID: \"1d075c21-db31-400b-b173-02391280ad9f\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"
Oct 30 22:19:50.747037 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:50.747016    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgzvm\" (UniqueName: \"kubernetes.io/projected/1d075c21-db31-400b-b173-02391280ad9f-kube-api-access-xgzvm\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8\" (UID: \"1d075c21-db31-400b-b173-02391280ad9f\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"
Oct 30 22:19:50.747276 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:50.747254    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d075c21-db31-400b-b173-02391280ad9f-istio-podinfo\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8\" (UID: \"1d075c21-db31-400b-b173-02391280ad9f\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"
Oct 30 22:19:50.758264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:50.750935    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d075c21-db31-400b-b173-02391280ad9f-istio-data\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8\" (UID: \"1d075c21-db31-400b-b173-02391280ad9f\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"
Oct 30 22:19:50.758970 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:50.758937    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d075c21-db31-400b-b173-02391280ad9f-istiod-ca-cert\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8\" (UID: \"1d075c21-db31-400b-b173-02391280ad9f\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"
Oct 30 22:19:50.759198 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:50.759180    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d075c21-db31-400b-b173-02391280ad9f-istio-envoy\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8\" (UID: \"1d075c21-db31-400b-b173-02391280ad9f\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"
Oct 30 22:19:50.759405 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:50.759387    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d075c21-db31-400b-b173-02391280ad9f-istio-token\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8\" (UID: \"1d075c21-db31-400b-b173-02391280ad9f\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"
Oct 30 22:19:50.759557 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:50.759540    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d075c21-db31-400b-b173-02391280ad9f-workload-socket\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8\" (UID: \"1d075c21-db31-400b-b173-02391280ad9f\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"
Oct 30 22:19:50.860479 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:50.860385    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d075c21-db31-400b-b173-02391280ad9f-istio-podinfo\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8\" (UID: \"1d075c21-db31-400b-b173-02391280ad9f\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"
Oct 30 22:19:50.860479 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:50.860443    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d075c21-db31-400b-b173-02391280ad9f-istio-data\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8\" (UID: \"1d075c21-db31-400b-b173-02391280ad9f\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"
Oct 30 22:19:50.860479 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:50.860470    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d075c21-db31-400b-b173-02391280ad9f-istiod-ca-cert\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8\" (UID: \"1d075c21-db31-400b-b173-02391280ad9f\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"
Oct 30 22:19:50.860794 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:50.860502    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d075c21-db31-400b-b173-02391280ad9f-istio-envoy\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8\" (UID: \"1d075c21-db31-400b-b173-02391280ad9f\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"
Oct 30 22:19:50.860794 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:50.860535    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d075c21-db31-400b-b173-02391280ad9f-istio-token\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8\" (UID: \"1d075c21-db31-400b-b173-02391280ad9f\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"
Oct 30 22:19:50.860794 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:50.860561    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d075c21-db31-400b-b173-02391280ad9f-workload-socket\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8\" (UID: \"1d075c21-db31-400b-b173-02391280ad9f\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"
Oct 30 22:19:50.860794 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:50.860609    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d075c21-db31-400b-b173-02391280ad9f-workload-certs\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8\" (UID: \"1d075c21-db31-400b-b173-02391280ad9f\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"
Oct 30 22:19:50.860794 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:50.860719    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d075c21-db31-400b-b173-02391280ad9f-credential-socket\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8\" (UID: \"1d075c21-db31-400b-b173-02391280ad9f\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"
Oct 30 22:19:50.860794 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:50.860783    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgzvm\" (UniqueName: \"kubernetes.io/projected/1d075c21-db31-400b-b173-02391280ad9f-kube-api-access-xgzvm\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8\" (UID: \"1d075c21-db31-400b-b173-02391280ad9f\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"
Oct 30 22:19:50.861115 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:50.860898    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d075c21-db31-400b-b173-02391280ad9f-istio-data\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8\" (UID: \"1d075c21-db31-400b-b173-02391280ad9f\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"
Oct 30 22:19:50.861115 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:50.861000    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d075c21-db31-400b-b173-02391280ad9f-workload-socket\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8\" (UID: \"1d075c21-db31-400b-b173-02391280ad9f\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"
Oct 30 22:19:50.861198 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:50.861180    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d075c21-db31-400b-b173-02391280ad9f-credential-socket\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8\" (UID: \"1d075c21-db31-400b-b173-02391280ad9f\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"
Oct 30 22:19:50.861273 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:50.861207    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d075c21-db31-400b-b173-02391280ad9f-workload-certs\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8\" (UID: \"1d075c21-db31-400b-b173-02391280ad9f\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"
Oct 30 22:19:50.861598 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:50.861579    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d075c21-db31-400b-b173-02391280ad9f-istiod-ca-cert\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8\" (UID: \"1d075c21-db31-400b-b173-02391280ad9f\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"
Oct 30 22:19:50.863335 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:50.863307    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d075c21-db31-400b-b173-02391280ad9f-istio-envoy\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8\" (UID: \"1d075c21-db31-400b-b173-02391280ad9f\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"
Oct 30 22:19:50.863469 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:50.863430    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d075c21-db31-400b-b173-02391280ad9f-istio-podinfo\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8\" (UID: \"1d075c21-db31-400b-b173-02391280ad9f\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"
Oct 30 22:19:50.928448 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:50.928411    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d075c21-db31-400b-b173-02391280ad9f-istio-token\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8\" (UID: \"1d075c21-db31-400b-b173-02391280ad9f\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"
Oct 30 22:19:50.928633 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:50.928565    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgzvm\" (UniqueName: \"kubernetes.io/projected/1d075c21-db31-400b-b173-02391280ad9f-kube-api-access-xgzvm\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8\" (UID: \"1d075c21-db31-400b-b173-02391280ad9f\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"
Oct 30 22:19:51.017409 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:51.017370    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:51.161766 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:51.161724    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"
Oct 30 22:19:51.593682 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:51.593591    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"]
Oct 30 22:19:51.624404 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:19:51.624371    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d075c21_db31_400b_b173_02391280ad9f.slice/crio-bbaf6e7d72ae8504527a3b44b9718f87baeb9874583403aec785bbfaffff21b2 WatchSource:0}: Error finding container bbaf6e7d72ae8504527a3b44b9718f87baeb9874583403aec785bbfaffff21b2: Status 404 returned error can't find the container with id bbaf6e7d72ae8504527a3b44b9718f87baeb9874583403aec785bbfaffff21b2
Oct 30 22:19:51.627360 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:51.627327    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:19:51.627462 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:51.627428    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:19:51.627506 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:51.627461    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:19:51.719217 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:51.719178    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8" event={"ID":"1d075c21-db31-400b-b173-02391280ad9f","Type":"ContainerStarted","Data":"c1115d043b1b29267fd73e98f53673914840a539e9549af2910422ba997edb30"}
Oct 30 22:19:51.719521 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:51.719479    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8" event={"ID":"1d075c21-db31-400b-b173-02391280ad9f","Type":"ContainerStarted","Data":"bbaf6e7d72ae8504527a3b44b9718f87baeb9874583403aec785bbfaffff21b2"}
Oct 30 22:19:51.836540 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:51.836470    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8" podStartSLOduration=1.836450648 podStartE2EDuration="1.836450648s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:19:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:19:51.83402532 +0000 UTC m=+2534.011513941" watchObservedRunningTime="2026-10-30 22:19:51.836450648 +0000 UTC m=+2534.013939266"
Oct 30 22:19:52.162661 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:52.162624    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"
Oct 30 22:19:52.169711 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:52.169686    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"
Oct 30 22:19:52.730368 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:52.730333    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"
Oct 30 22:19:52.731552 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:52.731526    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"
Oct 30 22:19:54.543109 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:54.543069    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:54.701194 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:54.701160    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:54.874819 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:54.874777    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:55.271028 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:55.270941    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:57.825923 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:57.825874    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:58.046284 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:58.046192    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:59.034524 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:59.034483    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:59.544649 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:19:59.544609    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:02.360423 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:02.360386    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:05.569656 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:20:05.569616    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:20:05.887448 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:05.887415    4871 generic.go:356] "Generic (PLEG): container finished" podID="1fad116b-e1cc-42da-a439-bdecb46b96a9" containerID="6925013eb32b2bc795b436ed113177007fa8629b8f2b226802e10ab405000376" exitCode=0
Oct 30 22:20:05.887675 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:05.887491    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb" event={"ID":"1fad116b-e1cc-42da-a439-bdecb46b96a9","Type":"ContainerDied","Data":"6925013eb32b2bc795b436ed113177007fa8629b8f2b226802e10ab405000376"}
Oct 30 22:20:05.887979 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:05.887961    4871 scope.go:122] "RemoveContainer" containerID="6925013eb32b2bc795b436ed113177007fa8629b8f2b226802e10ab405000376"
Oct 30 22:20:05.889501 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:05.889472    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:20:05.889588 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:05.889540    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:20:05.889588 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:05.889572    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:20:06.896405 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:06.896361    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb" event={"ID":"1fad116b-e1cc-42da-a439-bdecb46b96a9","Type":"ContainerStarted","Data":"41cec9c0dfdcd0be3f7cdeebbcafa2f7bbc1a8d31c75a8878ff8fb9d94734030"}
Oct 30 22:20:07.258257 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:07.258150    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"
Oct 30 22:20:07.259681 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:07.259643    4871 patch_prober.go:29] interesting pod/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.158:15021/healthz/ready\": dial tcp 10.133.0.158:15021: connect: connection refused" start-of-body=
Oct 30 22:20:07.259839 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:07.259718    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb" podUID="1fad116b-e1cc-42da-a439-bdecb46b96a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.158:15021/healthz/ready\": dial tcp 10.133.0.158:15021: connect: connection refused"
Oct 30 22:20:07.541547 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:07.541437    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:08.258322 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:08.258285    4871 patch_prober.go:29] interesting pod/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.158:15021/healthz/ready\": dial tcp 10.133.0.158:15021: connect: connection refused" start-of-body=
Oct 30 22:20:08.258744 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:08.258351    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb" podUID="1fad116b-e1cc-42da-a439-bdecb46b96a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.158:15021/healthz/ready\": dial tcp 10.133.0.158:15021: connect: connection refused"
Oct 30 22:20:09.258768 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:09.258730    4871 patch_prober.go:29] interesting pod/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.158:15021/healthz/ready\": dial tcp 10.133.0.158:15021: connect: connection refused" start-of-body=
Oct 30 22:20:09.259194 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:09.258794    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb" podUID="1fad116b-e1cc-42da-a439-bdecb46b96a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.158:15021/healthz/ready\": dial tcp 10.133.0.158:15021: connect: connection refused"
Oct 30 22:20:10.258822 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:10.258782    4871 patch_prober.go:29] interesting pod/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.158:15021/healthz/ready\": dial tcp 10.133.0.158:15021: connect: connection refused" start-of-body=
Oct 30 22:20:10.259334 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:10.258852    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb" podUID="1fad116b-e1cc-42da-a439-bdecb46b96a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.158:15021/healthz/ready\": dial tcp 10.133.0.158:15021: connect: connection refused"
Oct 30 22:20:10.974266 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:10.974213    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"
Oct 30 22:20:11.258623 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:11.258527    4871 patch_prober.go:29] interesting pod/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.158:15021/healthz/ready\": dial tcp 10.133.0.158:15021: connect: connection refused" start-of-body=
Oct 30 22:20:11.258623 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:11.258592    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb" podUID="1fad116b-e1cc-42da-a439-bdecb46b96a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.158:15021/healthz/ready\": dial tcp 10.133.0.158:15021: connect: connection refused"
Oct 30 22:20:12.264015 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:12.263980    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"
Oct 30 22:20:12.265154 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:12.265130    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"
Oct 30 22:20:12.395560 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:12.395525    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:12.570483 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:12.570392    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:12.724843 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:12.724806    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:17.481046 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:17.481011    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:17.652403 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:17.652352    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:17.897435 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:17.897400    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:22.332509 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:22.332468    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8"]
Oct 30 22:20:22.337932 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:22.337905    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8"
Oct 30 22:20:22.376162 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:22.376120    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2b31083-b29f-4a68-a20c-853287a23d6e-maas-api-tls\") pod \"maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8\" (UID: \"c2b31083-b29f-4a68-a20c-853287a23d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8"
Oct 30 22:20:22.376367 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:22.376189    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2b31083-b29f-4a68-a20c-853287a23d6e-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8\" (UID: \"c2b31083-b29f-4a68-a20c-853287a23d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8"
Oct 30 22:20:22.376430 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:22.376400    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2b31083-b29f-4a68-a20c-853287a23d6e-metrics-certs\") pod \"maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8\" (UID: \"c2b31083-b29f-4a68-a20c-853287a23d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8"
Oct 30 22:20:22.376480 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:22.376448    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdz72\" (UniqueName: \"kubernetes.io/projected/c2b31083-b29f-4a68-a20c-853287a23d6e-kube-api-access-wdz72\") pod \"maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8\" (UID: \"c2b31083-b29f-4a68-a20c-853287a23d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8"
Oct 30 22:20:22.388231 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:22.388199    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-a-f4a424-serving-cert\""
Oct 30 22:20:22.438812 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:22.438776    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8"]
Oct 30 22:20:22.477797 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:22.477759    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2b31083-b29f-4a68-a20c-853287a23d6e-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8\" (UID: \"c2b31083-b29f-4a68-a20c-853287a23d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8"
Oct 30 22:20:22.477987 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:22.477891    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2b31083-b29f-4a68-a20c-853287a23d6e-metrics-certs\") pod \"maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8\" (UID: \"c2b31083-b29f-4a68-a20c-853287a23d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8"
Oct 30 22:20:22.477987 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:22.477917    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdz72\" (UniqueName: \"kubernetes.io/projected/c2b31083-b29f-4a68-a20c-853287a23d6e-kube-api-access-wdz72\") pod \"maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8\" (UID: \"c2b31083-b29f-4a68-a20c-853287a23d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8"
Oct 30 22:20:22.477987 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:22.477964    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2b31083-b29f-4a68-a20c-853287a23d6e-maas-api-tls\") pod \"maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8\" (UID: \"c2b31083-b29f-4a68-a20c-853287a23d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8"
Oct 30 22:20:22.478166 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:20:22.478064    4871 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-praxis-a-f4a424-serving-cert: secret "maas-api-e2e-praxis-a-f4a424-serving-cert" not found
Oct 30 22:20:22.478166 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:20:22.478129    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2b31083-b29f-4a68-a20c-853287a23d6e-maas-api-tls podName:c2b31083-b29f-4a68-a20c-853287a23d6e nodeName:}" failed. No retries permitted until 2026-10-30 22:20:22.978110258 +0000 UTC m=+2565.155598854 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c2b31083-b29f-4a68-a20c-853287a23d6e-maas-api-tls") pod "maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8" (UID: "c2b31083-b29f-4a68-a20c-853287a23d6e") : secret "maas-api-e2e-praxis-a-f4a424-serving-cert" not found
Oct 30 22:20:22.478569 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:22.478535    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2b31083-b29f-4a68-a20c-853287a23d6e-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8\" (UID: \"c2b31083-b29f-4a68-a20c-853287a23d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8"
Oct 30 22:20:22.480658 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:22.480633    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2b31083-b29f-4a68-a20c-853287a23d6e-metrics-certs\") pod \"maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8\" (UID: \"c2b31083-b29f-4a68-a20c-853287a23d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8"
Oct 30 22:20:22.521509 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:22.521465    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:22.537919 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:22.537879    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdz72\" (UniqueName: \"kubernetes.io/projected/c2b31083-b29f-4a68-a20c-853287a23d6e-kube-api-access-wdz72\") pod \"maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8\" (UID: \"c2b31083-b29f-4a68-a20c-853287a23d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8"
Oct 30 22:20:22.680800 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:22.680763    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:22.836948 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:22.836910    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:23.003573 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:23.003471    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2b31083-b29f-4a68-a20c-853287a23d6e-maas-api-tls\") pod \"maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8\" (UID: \"c2b31083-b29f-4a68-a20c-853287a23d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8"
Oct 30 22:20:23.006140 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:23.006113    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2b31083-b29f-4a68-a20c-853287a23d6e-maas-api-tls\") pod \"maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8\" (UID: \"c2b31083-b29f-4a68-a20c-853287a23d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8"
Oct 30 22:20:23.251857 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:23.251812    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8"
Oct 30 22:20:23.628070 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:23.628034    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8"]
Oct 30 22:20:23.628597 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:20:23.628566    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2b31083_b29f_4a68_a20c_853287a23d6e.slice/crio-25388e45e969435b7749327111eca239786650c7578315c957ce2195c7b67a47 WatchSource:0}: Error finding container 25388e45e969435b7749327111eca239786650c7578315c957ce2195c7b67a47: Status 404 returned error can't find the container with id 25388e45e969435b7749327111eca239786650c7578315c957ce2195c7b67a47
Oct 30 22:20:23.630071 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:23.630043    4871 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:20:24.055735 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:24.055647    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8" event={"ID":"c2b31083-b29f-4a68-a20c-853287a23d6e","Type":"ContainerStarted","Data":"25388e45e969435b7749327111eca239786650c7578315c957ce2195c7b67a47"}
Oct 30 22:20:25.064999 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:25.064955    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8" event={"ID":"c2b31083-b29f-4a68-a20c-853287a23d6e","Type":"ContainerStarted","Data":"26ed7cc26b7706f493abe24c9c0a9f4527309fe588591048ddf4ab4c47d79faa"}
Oct 30 22:20:25.065468 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:25.065173    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8"
Oct 30 22:20:27.524407 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:27.524309    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:31.077272 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:31.077227    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8"
Oct 30 22:20:31.192231 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:31.192172    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8" podStartSLOduration=8.357599374 podStartE2EDuration="9.19215482s" totalImagesPullingTime="834.555446ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:20:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:20:25.211435103 +0000 UTC m=+2567.388923720" watchObservedRunningTime="2026-10-30 22:20:31.19215482 +0000 UTC m=+2573.369643487"
Oct 30 22:20:32.411747 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:32.411686    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:35.571852 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:20:35.571809    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:20:37.329151 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:37.329103    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:42.848307 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:42.848224    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:43.386594 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:43.386532    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:48.000604 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:48.000561    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:48.860135 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:48.860098    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:49.747629 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:49.747596    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:50.090199 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:50.090105    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:50.776012 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:50.627805    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"]
Oct 30 22:20:50.776012 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:50.655542    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"
Oct 30 22:20:50.776012 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:50.706107    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-b-85d00c-openshift-default-dockercfg-tsjwc\""
Oct 30 22:20:50.776012 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:50.741941    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-workload-socket\") pod \"e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg\" (UID: \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\") " pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"
Oct 30 22:20:50.776012 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:50.741991    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgqll\" (UniqueName: \"kubernetes.io/projected/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-kube-api-access-fgqll\") pod \"e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg\" (UID: \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\") " pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"
Oct 30 22:20:50.776012 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:50.742018    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-istio-data\") pod \"e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg\" (UID: \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\") " pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"
Oct 30 22:20:50.776012 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:50.742062    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-credential-socket\") pod \"e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg\" (UID: \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\") " pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"
Oct 30 22:20:50.776012 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:50.742926    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-istiod-ca-cert\") pod \"e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg\" (UID: \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\") " pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"
Oct 30 22:20:50.776012 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:50.743019    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-workload-certs\") pod \"e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg\" (UID: \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\") " pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"
Oct 30 22:20:50.776012 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:50.743121    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-istio-envoy\") pod \"e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg\" (UID: \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\") " pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"
Oct 30 22:20:50.776012 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:50.743147    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-istio-token\") pod \"e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg\" (UID: \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\") " pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"
Oct 30 22:20:50.776012 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:50.743189    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-istio-podinfo\") pod \"e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg\" (UID: \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\") " pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"
Oct 30 22:20:50.776012 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:50.748177    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:50.776012 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:50.772767    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"]
Oct 30 22:20:50.849913 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:50.844230    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-workload-certs\") pod \"e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg\" (UID: \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\") " pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"
Oct 30 22:20:50.849913 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:50.844383    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-istio-envoy\") pod \"e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg\" (UID: \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\") " pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"
Oct 30 22:20:50.849913 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:50.844413    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-istio-token\") pod \"e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg\" (UID: \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\") " pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"
Oct 30 22:20:50.849913 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:50.844471    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-istio-podinfo\") pod \"e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg\" (UID: \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\") " pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"
Oct 30 22:20:50.849913 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:50.844531    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-workload-socket\") pod \"e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg\" (UID: \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\") " pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"
Oct 30 22:20:50.849913 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:50.844562    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgqll\" (UniqueName: \"kubernetes.io/projected/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-kube-api-access-fgqll\") pod \"e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg\" (UID: \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\") " pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"
Oct 30 22:20:50.849913 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:50.844589    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-istio-data\") pod \"e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg\" (UID: \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\") " pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"
Oct 30 22:20:50.849913 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:50.844635    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-credential-socket\") pod \"e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg\" (UID: \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\") " pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"
Oct 30 22:20:50.849913 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:50.844705    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-istiod-ca-cert\") pod \"e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg\" (UID: \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\") " pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"
Oct 30 22:20:50.849913 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:50.845658    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-istiod-ca-cert\") pod \"e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg\" (UID: \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\") " pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"
Oct 30 22:20:50.849913 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:50.845955    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-workload-certs\") pod \"e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg\" (UID: \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\") " pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"
Oct 30 22:20:50.869931 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:50.869897    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-istio-podinfo\") pod \"e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg\" (UID: \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\") " pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"
Oct 30 22:20:50.870430 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:50.870409    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-workload-socket\") pod \"e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg\" (UID: \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\") " pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"
Oct 30 22:20:50.870764 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:50.870738    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-istio-envoy\") pod \"e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg\" (UID: \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\") " pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"
Oct 30 22:20:50.871087 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:50.871064    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-istio-data\") pod \"e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg\" (UID: \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\") " pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"
Oct 30 22:20:50.873366 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:50.871429    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-credential-socket\") pod \"e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg\" (UID: \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\") " pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"
Oct 30 22:20:50.942679 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:50.942650    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-istio-token\") pod \"e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg\" (UID: \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\") " pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"
Oct 30 22:20:50.943600 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:50.943569    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgqll\" (UniqueName: \"kubernetes.io/projected/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-kube-api-access-fgqll\") pod \"e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg\" (UID: \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\") " pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"
Oct 30 22:20:50.949152 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:50.949080    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:50.995311 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:50.995005    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"
Oct 30 22:20:51.334489 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:51.334400    4871 generic.go:356] "Generic (PLEG): container finished" podID="d6407e4e-8850-4c67-8ad5-93c7fde3ea55" containerID="ff5ec537f48ff74c1c24133feccb802e9c75d66e1b69adb617f0d0a9543743c3" exitCode=0
Oct 30 22:20:51.334489 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:51.334464    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5" event={"ID":"d6407e4e-8850-4c67-8ad5-93c7fde3ea55","Type":"ContainerDied","Data":"ff5ec537f48ff74c1c24133feccb802e9c75d66e1b69adb617f0d0a9543743c3"}
Oct 30 22:20:51.334964 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:51.334940    4871 scope.go:122] "RemoveContainer" containerID="ff5ec537f48ff74c1c24133feccb802e9c75d66e1b69adb617f0d0a9543743c3"
Oct 30 22:20:51.336903 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:51.336876    4871 generic.go:356] "Generic (PLEG): container finished" podID="4d334339-1d4b-421b-856e-c2a0d9f4737a" containerID="ad7e4c29bbccf337b323631161bab60909fe00c53884b029ad4567f3ea592ef6" exitCode=0
Oct 30 22:20:51.337015 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:51.336944    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g" event={"ID":"4d334339-1d4b-421b-856e-c2a0d9f4737a","Type":"ContainerDied","Data":"ad7e4c29bbccf337b323631161bab60909fe00c53884b029ad4567f3ea592ef6"}
Oct 30 22:20:51.337096 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:51.337051    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:20:51.337156 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:51.337138    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:20:51.337208 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:51.337186    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:20:51.337343 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:51.337326    4871 scope.go:122] "RemoveContainer" containerID="ad7e4c29bbccf337b323631161bab60909fe00c53884b029ad4567f3ea592ef6"
Oct 30 22:20:51.338930 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:51.338900    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:20:51.339013 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:51.338984    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:20:51.339052 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:51.339030    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:20:51.476279 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:51.475001    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"]
Oct 30 22:20:51.481994 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:51.481958    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:20:51.482165 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:51.482047    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:20:51.482165 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:51.482096    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:20:52.357261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:52.356929    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g" event={"ID":"4d334339-1d4b-421b-856e-c2a0d9f4737a","Type":"ContainerStarted","Data":"849ada11e535fa4bffd8d26586363c6193392257e1aab2a58b45119ea3f615de"}
Oct 30 22:20:52.364307 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:52.361352    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5" event={"ID":"d6407e4e-8850-4c67-8ad5-93c7fde3ea55","Type":"ContainerStarted","Data":"c5b02f4153df0425fe27f39a39c9bccc08b77151c916e9c551aa887059fd0092"}
Oct 30 22:20:52.365125 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:52.365071    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg" event={"ID":"dd4e1319-f9e3-46e5-856b-67e2f5139b6e","Type":"ContainerStarted","Data":"bc6288e15eeee83ffb3730f14e83e7848119d8cc2485c97cb77a8b37d041b781"}
Oct 30 22:20:52.365272 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:52.365156    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg" event={"ID":"dd4e1319-f9e3-46e5-856b-67e2f5139b6e","Type":"ContainerStarted","Data":"c7f0b9773e814c0c443c4b045966280db8d963624f1d780f2a0872d69268e3f5"}
Oct 30 22:20:52.956264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:52.955612    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"
Oct 30 22:20:52.960563 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:52.960510    4871 patch_prober.go:29] interesting pod/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused" start-of-body=
Oct 30 22:20:52.960921 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:52.960877    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5" podUID="d6407e4e-8850-4c67-8ad5-93c7fde3ea55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused"
Oct 30 22:20:52.995792 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:52.995748    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"
Oct 30 22:20:53.001320 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:53.001289    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"
Oct 30 22:20:53.139022 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:53.138961    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg" podStartSLOduration=3.138940941 podStartE2EDuration="3.138940941s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:20:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:20:52.738631743 +0000 UTC m=+2594.916120361" watchObservedRunningTime="2026-10-30 22:20:53.138940941 +0000 UTC m=+2595.316429561"
Oct 30 22:20:53.267327 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:53.267208    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:53.273297 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:53.273267    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"
Oct 30 22:20:53.274064 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:53.274021    4871 patch_prober.go:29] interesting pod/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.159:15021/healthz/ready\": dial tcp 10.133.0.159:15021: connect: connection refused" start-of-body=
Oct 30 22:20:53.274156 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:53.274084    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g" podUID="4d334339-1d4b-421b-856e-c2a0d9f4737a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.159:15021/healthz/ready\": dial tcp 10.133.0.159:15021: connect: connection refused"
Oct 30 22:20:53.396626 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:53.396576    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"
Oct 30 22:20:53.397534 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:53.397511    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"
Oct 30 22:20:53.425261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:53.424928    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:53.575350 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:53.575108    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:53.957891 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:53.957853    4871 patch_prober.go:29] interesting pod/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused" start-of-body=
Oct 30 22:20:53.958536 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:53.958500    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5" podUID="d6407e4e-8850-4c67-8ad5-93c7fde3ea55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused"
Oct 30 22:20:54.273284 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:54.272946    4871 patch_prober.go:29] interesting pod/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.159:15021/healthz/ready\": dial tcp 10.133.0.159:15021: connect: connection refused" start-of-body=
Oct 30 22:20:54.273284 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:54.273005    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g" podUID="4d334339-1d4b-421b-856e-c2a0d9f4737a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.159:15021/healthz/ready\": dial tcp 10.133.0.159:15021: connect: connection refused"
Oct 30 22:20:54.376551 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:54.376515    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"
Oct 30 22:20:54.957263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:54.956683    4871 patch_prober.go:29] interesting pod/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused" start-of-body=
Oct 30 22:20:54.957263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:54.956750    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5" podUID="d6407e4e-8850-4c67-8ad5-93c7fde3ea55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused"
Oct 30 22:20:55.272560 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:55.272438    4871 patch_prober.go:29] interesting pod/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.159:15021/healthz/ready\": dial tcp 10.133.0.159:15021: connect: connection refused" start-of-body=
Oct 30 22:20:55.272859 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:55.272824    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g" podUID="4d334339-1d4b-421b-856e-c2a0d9f4737a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.159:15021/healthz/ready\": dial tcp 10.133.0.159:15021: connect: connection refused"
Oct 30 22:20:55.516263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:55.509865    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:55.682895 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:55.682854    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:55.856542 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:55.856504    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:55.956170 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:55.956065    4871 patch_prober.go:29] interesting pod/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused" start-of-body=
Oct 30 22:20:55.956486 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:55.956439    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5" podUID="d6407e4e-8850-4c67-8ad5-93c7fde3ea55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused"
Oct 30 22:20:56.272792 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:56.272689    4871 patch_prober.go:29] interesting pod/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.159:15021/healthz/ready\": dial tcp 10.133.0.159:15021: connect: connection refused" start-of-body=
Oct 30 22:20:56.272792 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:56.272771    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g" podUID="4d334339-1d4b-421b-856e-c2a0d9f4737a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.159:15021/healthz/ready\": dial tcp 10.133.0.159:15021: connect: connection refused"
Oct 30 22:20:56.956648 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:56.956605    4871 patch_prober.go:29] interesting pod/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused" start-of-body=
Oct 30 22:20:56.956872 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:56.956671    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5" podUID="d6407e4e-8850-4c67-8ad5-93c7fde3ea55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused"
Oct 30 22:20:57.273307 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:57.273177    4871 patch_prober.go:29] interesting pod/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.159:15021/healthz/ready\": dial tcp 10.133.0.159:15021: connect: connection refused" start-of-body=
Oct 30 22:20:57.273307 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:57.273267    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g" podUID="4d334339-1d4b-421b-856e-c2a0d9f4737a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.159:15021/healthz/ready\": dial tcp 10.133.0.159:15021: connect: connection refused"
Oct 30 22:20:57.450563 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:57.450514    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"
Oct 30 22:20:57.963069 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:57.963041    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"
Oct 30 22:20:58.278257 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:58.278160    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"
Oct 30 22:20:58.279439 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:58.279415    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"
Oct 30 22:20:58.479139 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:58.479103    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"
Oct 30 22:20:58.879653 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:20:58.879614    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:04.786628 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:04.786591    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:04.969864 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:04.969815    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:05.570938 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:21:05.570900    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:21:05.754503 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:05.754461    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:10.608350 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:10.608286    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:15.276361 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:15.276321    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:20.020075 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:20.020015    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:21.695516 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:21.695419    4871 generic.go:356] "Generic (PLEG): container finished" podID="1d075c21-db31-400b-b173-02391280ad9f" containerID="c1115d043b1b29267fd73e98f53673914840a539e9549af2910422ba997edb30" exitCode=0
Oct 30 22:21:21.695516 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:21.695482    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8" event={"ID":"1d075c21-db31-400b-b173-02391280ad9f","Type":"ContainerDied","Data":"c1115d043b1b29267fd73e98f53673914840a539e9549af2910422ba997edb30"}
Oct 30 22:21:21.696031 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:21.695918    4871 scope.go:122] "RemoveContainer" containerID="c1115d043b1b29267fd73e98f53673914840a539e9549af2910422ba997edb30"
Oct 30 22:21:21.697511 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:21.697483    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:21:21.697600 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:21.697554    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:21:21.697600 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:21.697588    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:21:22.730611 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:22.730582    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"
Oct 30 22:21:22.736569 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:22.735426    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8" event={"ID":"1d075c21-db31-400b-b173-02391280ad9f","Type":"ContainerStarted","Data":"3df5dfb85d51185b3f122c9e766848737c76dc07154a01a0fb0f093f72a62067"}
Oct 30 22:21:23.161909 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:23.161864    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"
Oct 30 22:21:23.163527 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:23.163497    4871 patch_prober.go:29] interesting pod/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.162:15021/healthz/ready\": dial tcp 10.133.0.162:15021: connect: connection refused" start-of-body=
Oct 30 22:21:23.163664 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:23.163546    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8" podUID="1d075c21-db31-400b-b173-02391280ad9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.162:15021/healthz/ready\": dial tcp 10.133.0.162:15021: connect: connection refused"
Oct 30 22:21:23.893010 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:23.892974    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:24.162936 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:24.162835    4871 patch_prober.go:29] interesting pod/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.162:15021/healthz/ready\": dial tcp 10.133.0.162:15021: connect: connection refused" start-of-body=
Oct 30 22:21:24.162936 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:24.162898    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8" podUID="1d075c21-db31-400b-b173-02391280ad9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.162:15021/healthz/ready\": dial tcp 10.133.0.162:15021: connect: connection refused"
Oct 30 22:21:25.163153 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:25.163113    4871 patch_prober.go:29] interesting pod/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.162:15021/healthz/ready\": dial tcp 10.133.0.162:15021: connect: connection refused" start-of-body=
Oct 30 22:21:25.163604 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:25.163172    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8" podUID="1d075c21-db31-400b-b173-02391280ad9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.162:15021/healthz/ready\": dial tcp 10.133.0.162:15021: connect: connection refused"
Oct 30 22:21:25.258714 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:25.258664    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:25.427994 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:25.427887    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:25.883109 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:25.883039    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:26.088257 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:26.088181    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:26.162729 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:26.162627    4871 patch_prober.go:29] interesting pod/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.162:15021/healthz/ready\": dial tcp 10.133.0.162:15021: connect: connection refused" start-of-body=
Oct 30 22:21:26.162729 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:26.162694    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8" podUID="1d075c21-db31-400b-b173-02391280ad9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.162:15021/healthz/ready\": dial tcp 10.133.0.162:15021: connect: connection refused"
Oct 30 22:21:27.162824 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:27.162787    4871 patch_prober.go:29] interesting pod/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.162:15021/healthz/ready\": dial tcp 10.133.0.162:15021: connect: connection refused" start-of-body=
Oct 30 22:21:27.163357 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:27.162851    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8" podUID="1d075c21-db31-400b-b173-02391280ad9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.162:15021/healthz/ready\": dial tcp 10.133.0.162:15021: connect: connection refused"
Oct 30 22:21:28.167359 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:28.167324    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"
Oct 30 22:21:28.167890 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:28.167867    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"
Oct 30 22:21:28.168643 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:28.168623    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"
Oct 30 22:21:30.278665 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:30.278615    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:31.180789 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:31.180751    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:31.410093 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:31.410038    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:31.610632 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:31.610544    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:32.076428 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:32.076394    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:35.255439 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:35.255405    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:35.572430 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:21:35.572330    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:21:35.954769 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:35.954732    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:36.298487 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:36.297927    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:36.477830 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:36.477786    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:36.600312 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:36.594491    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-fc3918-585f79b757-nzmct"]
Oct 30 22:21:36.628647 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:36.628613    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-fc3918-585f79b757-nzmct"
Oct 30 22:21:36.667800 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:36.666279    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-fc3918-585f79b757-nzmct"]
Oct 30 22:21:36.695507 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:36.695466    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rc5g\" (UniqueName: \"kubernetes.io/projected/914386df-d6fd-4b9d-92b9-45a987e3b514-kube-api-access-8rc5g\") pod \"payload-processing-e2e-praxis-b-fc3918-585f79b757-nzmct\" (UID: \"914386df-d6fd-4b9d-92b9-45a987e3b514\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-fc3918-585f79b757-nzmct"
Oct 30 22:21:36.695723 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:36.695693    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/914386df-d6fd-4b9d-92b9-45a987e3b514-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-fc3918-585f79b757-nzmct\" (UID: \"914386df-d6fd-4b9d-92b9-45a987e3b514\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-fc3918-585f79b757-nzmct"
Oct 30 22:21:36.735625 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:36.735579    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-b-fc3918\""
Oct 30 22:21:36.735625 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:36.735622    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-praxis-b-fc3918-dockercfg-z62ch\""
Oct 30 22:21:36.796831 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:36.796792    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/914386df-d6fd-4b9d-92b9-45a987e3b514-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-fc3918-585f79b757-nzmct\" (UID: \"914386df-d6fd-4b9d-92b9-45a987e3b514\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-fc3918-585f79b757-nzmct"
Oct 30 22:21:36.797022 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:36.796862    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rc5g\" (UniqueName: \"kubernetes.io/projected/914386df-d6fd-4b9d-92b9-45a987e3b514-kube-api-access-8rc5g\") pod \"payload-processing-e2e-praxis-b-fc3918-585f79b757-nzmct\" (UID: \"914386df-d6fd-4b9d-92b9-45a987e3b514\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-fc3918-585f79b757-nzmct"
Oct 30 22:21:36.797588 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:36.797567    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/914386df-d6fd-4b9d-92b9-45a987e3b514-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-fc3918-585f79b757-nzmct\" (UID: \"914386df-d6fd-4b9d-92b9-45a987e3b514\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-fc3918-585f79b757-nzmct"
Oct 30 22:21:36.858689 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:36.858610    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rc5g\" (UniqueName: \"kubernetes.io/projected/914386df-d6fd-4b9d-92b9-45a987e3b514-kube-api-access-8rc5g\") pod \"payload-processing-e2e-praxis-b-fc3918-585f79b757-nzmct\" (UID: \"914386df-d6fd-4b9d-92b9-45a987e3b514\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-fc3918-585f79b757-nzmct"
Oct 30 22:21:36.973855 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:36.973812    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-fc3918-585f79b757-nzmct"
Oct 30 22:21:37.352253 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:37.352201    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-fc3918-585f79b757-nzmct"]
Oct 30 22:21:37.353403 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:21:37.353374    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod914386df_d6fd_4b9d_92b9_45a987e3b514.slice/crio-8851a47a9a5c8f656308913965c155e360c08666577c01873c6751ba67830e79 WatchSource:0}: Error finding container 8851a47a9a5c8f656308913965c155e360c08666577c01873c6751ba67830e79: Status 404 returned error can't find the container with id 8851a47a9a5c8f656308913965c155e360c08666577c01873c6751ba67830e79
Oct 30 22:21:37.491676 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:37.491644    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:37.661271 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:37.656848    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:37.870844 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:37.870804    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-fc3918-585f79b757-nzmct" event={"ID":"914386df-d6fd-4b9d-92b9-45a987e3b514","Type":"ContainerStarted","Data":"4eca8ba18babdd0d2848d35422e760cdb90315d7dbc04372d2e35a15b740b291"}
Oct 30 22:21:37.871066 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:37.870866    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-fc3918-585f79b757-nzmct" event={"ID":"914386df-d6fd-4b9d-92b9-45a987e3b514","Type":"ContainerStarted","Data":"8851a47a9a5c8f656308913965c155e360c08666577c01873c6751ba67830e79"}
Oct 30 22:21:37.871066 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:37.871004    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-praxis-b-fc3918-585f79b757-nzmct"
Oct 30 22:21:37.986214 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:37.986092    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-praxis-b-fc3918-585f79b757-nzmct" podStartSLOduration=1.9860714590000002 podStartE2EDuration="1.986071459s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:21:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:21:37.982061649 +0000 UTC m=+2640.159550267" watchObservedRunningTime="2026-10-30 22:21:37.986071459 +0000 UTC m=+2640.163560080"
Oct 30 22:21:40.121057 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:40.121011    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:40.387416 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:40.387322    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:43.314449 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:43.314406    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:43.740740 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:43.740702    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:43.848180 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:43.848142    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"]
Oct 30 22:21:43.851655 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:43.851626    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"
Oct 30 22:21:43.904511 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:43.904388    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-b-93131d-openshift-default-dockercfg-mg4jh\""
Oct 30 22:21:43.905583 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:43.905552    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-workload-certs\") pod \"e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9\" (UID: \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\") " pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"
Oct 30 22:21:43.906378 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:43.906349    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-istio-data\") pod \"e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9\" (UID: \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\") " pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"
Oct 30 22:21:43.906576 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:43.906557    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4brw\" (UniqueName: \"kubernetes.io/projected/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-kube-api-access-t4brw\") pod \"e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9\" (UID: \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\") " pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"
Oct 30 22:21:43.906722 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:43.906706    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-workload-socket\") pod \"e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9\" (UID: \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\") " pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"
Oct 30 22:21:43.906850 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:43.906836    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-istio-envoy\") pod \"e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9\" (UID: \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\") " pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"
Oct 30 22:21:43.907012 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:43.906997    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-credential-socket\") pod \"e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9\" (UID: \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\") " pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"
Oct 30 22:21:43.907150 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:43.907135    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-istiod-ca-cert\") pod \"e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9\" (UID: \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\") " pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"
Oct 30 22:21:43.907277 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:43.907261    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-istio-token\") pod \"e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9\" (UID: \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\") " pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"
Oct 30 22:21:43.907460 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:43.907444    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-istio-podinfo\") pod \"e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9\" (UID: \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\") " pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"
Oct 30 22:21:43.988125 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:43.988073    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:44.061324 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:44.061049    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-istiod-ca-cert\") pod \"e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9\" (UID: \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\") " pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"
Oct 30 22:21:44.061504 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:44.061330    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-istio-token\") pod \"e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9\" (UID: \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\") " pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"
Oct 30 22:21:44.061504 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:44.061426    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-istio-podinfo\") pod \"e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9\" (UID: \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\") " pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"
Oct 30 22:21:44.061639 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:44.061499    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-workload-certs\") pod \"e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9\" (UID: \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\") " pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"
Oct 30 22:21:44.061639 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:44.061586    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-istio-data\") pod \"e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9\" (UID: \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\") " pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"
Oct 30 22:21:44.061752 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:44.061634    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4brw\" (UniqueName: \"kubernetes.io/projected/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-kube-api-access-t4brw\") pod \"e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9\" (UID: \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\") " pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"
Oct 30 22:21:44.061752 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:44.061680    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-workload-socket\") pod \"e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9\" (UID: \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\") " pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"
Oct 30 22:21:44.061752 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:44.061724    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-istio-envoy\") pod \"e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9\" (UID: \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\") " pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"
Oct 30 22:21:44.061906 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:44.061811    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-credential-socket\") pod \"e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9\" (UID: \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\") " pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"
Oct 30 22:21:44.062494 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:44.062116    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-istiod-ca-cert\") pod \"e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9\" (UID: \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\") " pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"
Oct 30 22:21:44.062494 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:44.062254    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-credential-socket\") pod \"e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9\" (UID: \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\") " pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"
Oct 30 22:21:44.062494 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:44.062311    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-istio-data\") pod \"e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9\" (UID: \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\") " pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"
Oct 30 22:21:44.062494 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:44.062456    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-workload-socket\") pod \"e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9\" (UID: \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\") " pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"
Oct 30 22:21:44.063179 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:44.063133    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-workload-certs\") pod \"e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9\" (UID: \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\") " pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"
Oct 30 22:21:44.063989 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:44.063963    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"]
Oct 30 22:21:44.064973 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:44.064951    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-istio-envoy\") pod \"e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9\" (UID: \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\") " pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"
Oct 30 22:21:44.065648 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:44.065619    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-istio-podinfo\") pod \"e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9\" (UID: \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\") " pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"
Oct 30 22:21:44.121525 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:44.121493    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4brw\" (UniqueName: \"kubernetes.io/projected/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-kube-api-access-t4brw\") pod \"e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9\" (UID: \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\") " pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"
Oct 30 22:21:44.121694 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:44.121496    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-istio-token\") pod \"e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9\" (UID: \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\") " pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"
Oct 30 22:21:44.165710 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:44.165669    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"
Oct 30 22:21:44.305996 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:44.305929    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:44.633612 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:44.633566    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"]
Oct 30 22:21:44.654443 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:21:44.654404    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod689c9d7f_c8ab_49e5_b647_b4a62d97c1e7.slice/crio-5143f3600708be81db52c4ef054bea4a21100bcf3f0a125aba69748cb686fe85 WatchSource:0}: Error finding container 5143f3600708be81db52c4ef054bea4a21100bcf3f0a125aba69748cb686fe85: Status 404 returned error can't find the container with id 5143f3600708be81db52c4ef054bea4a21100bcf3f0a125aba69748cb686fe85
Oct 30 22:21:44.657136 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:44.657101    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:21:44.657208 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:44.657181    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:21:44.657284 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:44.657216    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:21:44.760187 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:44.760148    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:44.998639 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:44.998541    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9" event={"ID":"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7","Type":"ContainerStarted","Data":"236a45bbecdb97b8c9ab767753bc6e8ed506280be126a9fb0cf863f326e843a5"}
Oct 30 22:21:44.998639 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:44.998600    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9" event={"ID":"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7","Type":"ContainerStarted","Data":"5143f3600708be81db52c4ef054bea4a21100bcf3f0a125aba69748cb686fe85"}
Oct 30 22:21:45.114593 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:45.114532    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9" podStartSLOduration=2.114519219 podStartE2EDuration="2.114519219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:21:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:21:45.114291836 +0000 UTC m=+2647.291780452" watchObservedRunningTime="2026-10-30 22:21:45.114519219 +0000 UTC m=+2647.292007878"
Oct 30 22:21:45.166592 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:45.166548    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"
Oct 30 22:21:45.172644 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:45.172614    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"
Oct 30 22:21:46.006184 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:46.006141    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"
Oct 30 22:21:46.007192 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:46.007174    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"
Oct 30 22:21:47.800475 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:47.800432    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:47.966391 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:47.966294    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:48.142605 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:48.142568    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:48.888321 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:48.888290    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-praxis-b-fc3918-585f79b757-nzmct"
Oct 30 22:21:49.656541 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:49.656496    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:51.019184 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:51.019145    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:51.224491 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:51.224460    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:53.168222 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:53.168151    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:54.648759 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:54.648722    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:59.260665 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:59.260628    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:59.424480 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:59.424440    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:59.578484 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:21:59.578395    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:01.803959 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:01.803926    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:03.058928 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:03.058894    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:03.571850 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:03.571807    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"]
Oct 30 22:22:03.577068 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:03.577044    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"
Oct 30 22:22:03.647778 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:03.647741    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-b-7fd427-openshift-default-dockercfg-qk6k2\""
Oct 30 22:22:03.653556 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:03.653519    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-istiod-ca-cert\") pod \"e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz\" (UID: \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\") " pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"
Oct 30 22:22:03.653836 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:03.653807    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-istio-podinfo\") pod \"e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz\" (UID: \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\") " pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"
Oct 30 22:22:03.654098 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:03.653852    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-istio-token\") pod \"e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz\" (UID: \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\") " pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"
Oct 30 22:22:03.654098 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:03.653938    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl2wq\" (UniqueName: \"kubernetes.io/projected/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-kube-api-access-kl2wq\") pod \"e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz\" (UID: \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\") " pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"
Oct 30 22:22:03.654098 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:03.653993    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-istio-data\") pod \"e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz\" (UID: \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\") " pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"
Oct 30 22:22:03.654098 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:03.654036    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-credential-socket\") pod \"e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz\" (UID: \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\") " pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"
Oct 30 22:22:03.654098 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:03.654074    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-istio-envoy\") pod \"e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz\" (UID: \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\") " pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"
Oct 30 22:22:03.654432 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:03.654137    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-workload-certs\") pod \"e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz\" (UID: \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\") " pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"
Oct 30 22:22:03.654432 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:03.654256    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-workload-socket\") pod \"e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz\" (UID: \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\") " pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"
Oct 30 22:22:03.676986 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:03.676950    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"]
Oct 30 22:22:03.709502 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:03.709455    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:03.766273 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:03.758850    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-workload-socket\") pod \"e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz\" (UID: \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\") " pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"
Oct 30 22:22:03.766562 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:03.766535    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-istiod-ca-cert\") pod \"e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz\" (UID: \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\") " pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"
Oct 30 22:22:03.772391 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:03.772358    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-istio-podinfo\") pod \"e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz\" (UID: \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\") " pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"
Oct 30 22:22:03.772612 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:03.772592    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-istio-token\") pod \"e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz\" (UID: \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\") " pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"
Oct 30 22:22:03.772829 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:03.772806    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl2wq\" (UniqueName: \"kubernetes.io/projected/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-kube-api-access-kl2wq\") pod \"e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz\" (UID: \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\") " pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"
Oct 30 22:22:03.773009 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:03.772992    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-istio-data\") pod \"e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz\" (UID: \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\") " pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"
Oct 30 22:22:03.773153 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:03.773137    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-credential-socket\") pod \"e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz\" (UID: \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\") " pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"
Oct 30 22:22:03.780359 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:03.780330    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-istio-envoy\") pod \"e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz\" (UID: \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\") " pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"
Oct 30 22:22:03.780629 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:03.780605    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-workload-certs\") pod \"e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz\" (UID: \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\") " pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"
Oct 30 22:22:03.780925 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:03.780899    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-workload-certs\") pod \"e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz\" (UID: \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\") " pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"
Oct 30 22:22:03.781084 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:03.774416    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-credential-socket\") pod \"e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz\" (UID: \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\") " pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"
Oct 30 22:22:03.781195 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:03.761763    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-workload-socket\") pod \"e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz\" (UID: \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\") " pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"
Oct 30 22:22:03.781322 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:03.774168    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-istio-data\") pod \"e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz\" (UID: \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\") " pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"
Oct 30 22:22:03.781435 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:03.773015    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-istiod-ca-cert\") pod \"e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz\" (UID: \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\") " pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"
Oct 30 22:22:03.786630 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:03.786605    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-istio-podinfo\") pod \"e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz\" (UID: \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\") " pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"
Oct 30 22:22:03.847404 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:03.847271    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl2wq\" (UniqueName: \"kubernetes.io/projected/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-kube-api-access-kl2wq\") pod \"e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz\" (UID: \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\") " pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"
Oct 30 22:22:03.851868 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:03.851796    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-istio-token\") pod \"e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz\" (UID: \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\") " pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"
Oct 30 22:22:03.863271 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:03.855507    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-istio-envoy\") pod \"e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz\" (UID: \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\") " pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"
Oct 30 22:22:03.894349 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:03.894315    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"
Oct 30 22:22:03.968824 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:03.968695    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:04.356317 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:04.356269    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"]
Oct 30 22:22:04.357474 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:22:04.357436    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4aaa6ee_3ee7_44f4_93ab_673837d4a23b.slice/crio-d6dd37bebac94eaf967c4351cee1bb278601b10b4cfa9bc70a4a75aba9b33f45 WatchSource:0}: Error finding container d6dd37bebac94eaf967c4351cee1bb278601b10b4cfa9bc70a4a75aba9b33f45: Status 404 returned error can't find the container with id d6dd37bebac94eaf967c4351cee1bb278601b10b4cfa9bc70a4a75aba9b33f45
Oct 30 22:22:04.359981 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:04.359949    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:22:04.360067 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:04.360013    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:22:04.360067 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:04.360043    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:22:04.731773 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:04.731721    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:05.167940 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:05.167900    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz" event={"ID":"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b","Type":"ContainerStarted","Data":"446a7c884a4280e9cfbcd0dfc303c3f87bafa507bc5c657fde1ecfeba6339ab8"}
Oct 30 22:22:05.168126 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:05.167958    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz" event={"ID":"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b","Type":"ContainerStarted","Data":"d6dd37bebac94eaf967c4351cee1bb278601b10b4cfa9bc70a4a75aba9b33f45"}
Oct 30 22:22:05.285015 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:05.284950    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz" podStartSLOduration=2.2849301 podStartE2EDuration="2.2849301s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:22:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:22:05.282805781 +0000 UTC m=+2667.460294397" watchObservedRunningTime="2026-10-30 22:22:05.2849301 +0000 UTC m=+2667.462418717"
Oct 30 22:22:05.569407 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:22:05.569323    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:22:05.895678 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:05.895645    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"
Oct 30 22:22:05.901229 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:05.901201    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"
Oct 30 22:22:06.174867 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:06.174776    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"
Oct 30 22:22:06.175921 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:06.175897    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"
Oct 30 22:22:10.906381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:10.906342    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:13.783582 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:13.783551    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:14.735657 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:14.735620    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:15.414746 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:15.414708    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:17.938974 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:17.938939    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:18.132085 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:18.132042    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:18.302703 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:18.302606    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:20.214387 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:20.214342    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:21.205351 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:21.205309    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:21.396277 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:21.396191    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:25.196516 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:25.196480    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:28.638851 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:28.638807    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:29.128682 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:29.128649    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:31.648844 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:31.648791    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:35.577626 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:22:35.577582    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:22:36.153817 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:36.153783    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:36.310784 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:36.310743    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:36.443997 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:36.443902    4871 generic.go:356] "Generic (PLEG): container finished" podID="dd4e1319-f9e3-46e5-856b-67e2f5139b6e" containerID="bc6288e15eeee83ffb3730f14e83e7848119d8cc2485c97cb77a8b37d041b781" exitCode=0
Oct 30 22:22:36.444186 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:36.443990    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg" event={"ID":"dd4e1319-f9e3-46e5-856b-67e2f5139b6e","Type":"ContainerDied","Data":"bc6288e15eeee83ffb3730f14e83e7848119d8cc2485c97cb77a8b37d041b781"}
Oct 30 22:22:36.444512 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:36.444493    4871 scope.go:122] "RemoveContainer" containerID="bc6288e15eeee83ffb3730f14e83e7848119d8cc2485c97cb77a8b37d041b781"
Oct 30 22:22:36.446509 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:36.446471    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:22:36.446613 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:36.446565    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:22:36.446613 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:36.446607    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:22:36.460429 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:36.460391    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:37.460842 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:37.460803    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg" event={"ID":"dd4e1319-f9e3-46e5-856b-67e2f5139b6e","Type":"ContainerStarted","Data":"24cc7199824b1069243be9669a8ba55fa9058ed60035991411cfa5789f999b73"}
Oct 30 22:22:37.996219 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:37.996186    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"
Oct 30 22:22:37.997270 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:37.997230    4871 patch_prober.go:29] interesting pod/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.164:15021/healthz/ready\": dial tcp 10.133.0.164:15021: connect: connection refused" start-of-body=
Oct 30 22:22:37.997340 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:37.997295    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg" podUID="dd4e1319-f9e3-46e5-856b-67e2f5139b6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.164:15021/healthz/ready\": dial tcp 10.133.0.164:15021: connect: connection refused"
Oct 30 22:22:38.396704 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:38.396663    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"
Oct 30 22:22:38.492022 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:38.491997    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx_8d401654-8a4c-4a79-b2d3-612765312421/extract/0.log"
Oct 30 22:22:38.494441 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:38.494411    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx_8d401654-8a4c-4a79-b2d3-612765312421/extract/0.log"
Oct 30 22:22:38.536289 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:38.536170    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/2.log"
Oct 30 22:22:38.539555 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:38.539535    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/2.log"
Oct 30 22:22:38.546125 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:38.546097    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzntd_2360fddc-52a5-42a5-82ca-a05aff84d217/ovn-acl-logging/0.log"
Oct 30 22:22:38.548759 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:38.548741    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzntd_2360fddc-52a5-42a5-82ca-a05aff84d217/ovn-acl-logging/0.log"
Oct 30 22:22:38.999261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:38.997969    4871 patch_prober.go:29] interesting pod/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.164:15021/healthz/ready\": dial tcp 10.133.0.164:15021: connect: connection refused" start-of-body=
Oct 30 22:22:38.999261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:38.998040    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg" podUID="dd4e1319-f9e3-46e5-856b-67e2f5139b6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.164:15021/healthz/ready\": dial tcp 10.133.0.164:15021: connect: connection refused"
Oct 30 22:22:39.996484 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:39.996447    4871 patch_prober.go:29] interesting pod/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.164:15021/healthz/ready\": dial tcp 10.133.0.164:15021: connect: connection refused" start-of-body=
Oct 30 22:22:39.996916 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:39.996505    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg" podUID="dd4e1319-f9e3-46e5-856b-67e2f5139b6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.164:15021/healthz/ready\": dial tcp 10.133.0.164:15021: connect: connection refused"
Oct 30 22:22:40.996024 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:40.995986    4871 patch_prober.go:29] interesting pod/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.164:15021/healthz/ready\": dial tcp 10.133.0.164:15021: connect: connection refused" start-of-body=
Oct 30 22:22:40.996262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:40.996062    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg" podUID="dd4e1319-f9e3-46e5-856b-67e2f5139b6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.164:15021/healthz/ready\": dial tcp 10.133.0.164:15021: connect: connection refused"
Oct 30 22:22:41.032298 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:41.021542    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:41.996923 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:41.996883    4871 patch_prober.go:29] interesting pod/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.164:15021/healthz/ready\": dial tcp 10.133.0.164:15021: connect: connection refused" start-of-body=
Oct 30 22:22:41.997156 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:41.996947    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg" podUID="dd4e1319-f9e3-46e5-856b-67e2f5139b6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.164:15021/healthz/ready\": dial tcp 10.133.0.164:15021: connect: connection refused"
Oct 30 22:22:43.051403 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:43.051357    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"
Oct 30 22:22:43.053913 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:43.053883    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"
Oct 30 22:22:44.077357 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:44.077320    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:44.331733 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:44.331642    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:45.613745 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:45.613700    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:50.137817 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:50.137780    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:54.795263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:54.793044    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:58.981072 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:58.981030    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2"]
Oct 30 22:22:58.985282 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:58.985232    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2"
Oct 30 22:22:59.004711 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:59.004680    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d03255e-3722-45d9-b21b-b1c751355173-maas-api-tls\") pod \"maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2\" (UID: \"5d03255e-3722-45d9-b21b-b1c751355173\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2"
Oct 30 22:22:59.004888 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:59.004858    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d03255e-3722-45d9-b21b-b1c751355173-metrics-certs\") pod \"maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2\" (UID: \"5d03255e-3722-45d9-b21b-b1c751355173\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2"
Oct 30 22:22:59.004944 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:59.004928    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d03255e-3722-45d9-b21b-b1c751355173-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2\" (UID: \"5d03255e-3722-45d9-b21b-b1c751355173\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2"
Oct 30 22:22:59.004989 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:59.004974    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nk4x6\" (UniqueName: \"kubernetes.io/projected/5d03255e-3722-45d9-b21b-b1c751355173-kube-api-access-nk4x6\") pod \"maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2\" (UID: \"5d03255e-3722-45d9-b21b-b1c751355173\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2"
Oct 30 22:22:59.036166 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:59.036132    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-b-93131d-serving-cert\""
Oct 30 22:22:59.087675 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:59.087633    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2"]
Oct 30 22:22:59.106774 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:59.106715    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d03255e-3722-45d9-b21b-b1c751355173-maas-api-tls\") pod \"maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2\" (UID: \"5d03255e-3722-45d9-b21b-b1c751355173\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2"
Oct 30 22:22:59.107149 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:22:59.107034    4871 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-praxis-b-93131d-serving-cert: secret "maas-api-e2e-praxis-b-93131d-serving-cert" not found
Oct 30 22:22:59.107275 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:59.107191    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d03255e-3722-45d9-b21b-b1c751355173-metrics-certs\") pod \"maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2\" (UID: \"5d03255e-3722-45d9-b21b-b1c751355173\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2"
Oct 30 22:22:59.107275 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:22:59.107218    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d03255e-3722-45d9-b21b-b1c751355173-maas-api-tls podName:5d03255e-3722-45d9-b21b-b1c751355173 nodeName:}" failed. No retries permitted until 2026-10-30 22:22:59.607192221 +0000 UTC m=+2721.784680819 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5d03255e-3722-45d9-b21b-b1c751355173-maas-api-tls") pod "maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2" (UID: "5d03255e-3722-45d9-b21b-b1c751355173") : secret "maas-api-e2e-praxis-b-93131d-serving-cert" not found
Oct 30 22:22:59.107409 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:59.107304    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d03255e-3722-45d9-b21b-b1c751355173-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2\" (UID: \"5d03255e-3722-45d9-b21b-b1c751355173\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2"
Oct 30 22:22:59.107409 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:59.107355    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nk4x6\" (UniqueName: \"kubernetes.io/projected/5d03255e-3722-45d9-b21b-b1c751355173-kube-api-access-nk4x6\") pod \"maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2\" (UID: \"5d03255e-3722-45d9-b21b-b1c751355173\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2"
Oct 30 22:22:59.108430 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:59.108406    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d03255e-3722-45d9-b21b-b1c751355173-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2\" (UID: \"5d03255e-3722-45d9-b21b-b1c751355173\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2"
Oct 30 22:22:59.116276 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:59.110294    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d03255e-3722-45d9-b21b-b1c751355173-metrics-certs\") pod \"maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2\" (UID: \"5d03255e-3722-45d9-b21b-b1c751355173\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2"
Oct 30 22:22:59.166046 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:59.166008    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nk4x6\" (UniqueName: \"kubernetes.io/projected/5d03255e-3722-45d9-b21b-b1c751355173-kube-api-access-nk4x6\") pod \"maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2\" (UID: \"5d03255e-3722-45d9-b21b-b1c751355173\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2"
Oct 30 22:22:59.195385 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:59.195342    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:59.346776 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:59.346686    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:59.614985 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:59.614942    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d03255e-3722-45d9-b21b-b1c751355173-maas-api-tls\") pod \"maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2\" (UID: \"5d03255e-3722-45d9-b21b-b1c751355173\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2"
Oct 30 22:22:59.617706 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:59.617677    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d03255e-3722-45d9-b21b-b1c751355173-maas-api-tls\") pod \"maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2\" (UID: \"5d03255e-3722-45d9-b21b-b1c751355173\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2"
Oct 30 22:22:59.898294 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:22:59.898187    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2"
Oct 30 22:23:00.286112 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:00.286073    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2"]
Oct 30 22:23:00.286511 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:23:00.286326    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d03255e_3722_45d9_b21b_b1c751355173.slice/crio-15f641577b849fbc6eb9e5f43aed8261903ae158c0c4a93ab8437d339c9c9cb8 WatchSource:0}: Error finding container 15f641577b849fbc6eb9e5f43aed8261903ae158c0c4a93ab8437d339c9c9cb8: Status 404 returned error can't find the container with id 15f641577b849fbc6eb9e5f43aed8261903ae158c0c4a93ab8437d339c9c9cb8
Oct 30 22:23:00.641491 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:00.641440    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2" event={"ID":"5d03255e-3722-45d9-b21b-b1c751355173","Type":"ContainerStarted","Data":"15f641577b849fbc6eb9e5f43aed8261903ae158c0c4a93ab8437d339c9c9cb8"}
Oct 30 22:23:01.652710 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:01.652414    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2" event={"ID":"5d03255e-3722-45d9-b21b-b1c751355173","Type":"ContainerStarted","Data":"990490fa7c4094eb832fbf2b1722244814add3160ec289dd8a6bcdaab41509e7"}
Oct 30 22:23:01.652710 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:01.652599    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2"
Oct 30 22:23:01.774959 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:01.770088    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2" podStartSLOduration=2.793516318 podStartE2EDuration="3.770068851s" totalImagesPullingTime="976.552533ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:22:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:23:01.765021617 +0000 UTC m=+2723.942510233" watchObservedRunningTime="2026-10-30 22:23:01.770068851 +0000 UTC m=+2723.947557469"
Oct 30 22:23:04.045763 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:04.045721    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:05.569704 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:23:05.569663    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:23:07.666464 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:07.666431    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2"
Oct 30 22:23:08.688261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:08.688204    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:13.212445 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:13.212413    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:18.042942 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:18.042844    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:24.496247 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:24.496210    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:28.329278 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:28.325013    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:29.173790 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:29.173724    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:32.047530 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:32.047480    4871 patch_prober.go:29] interesting pod/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.166:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:23:32.047974 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:32.047577    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9" podUID="689c9d7f-c8ab-49e5-b647-b4a62d97c1e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.166:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:23:32.909330 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:32.909200    4871 generic.go:356] "Generic (PLEG): container finished" podID="689c9d7f-c8ab-49e5-b647-b4a62d97c1e7" containerID="236a45bbecdb97b8c9ab767753bc6e8ed506280be126a9fb0cf863f326e843a5" exitCode=0
Oct 30 22:23:32.909330 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:32.909272    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9" event={"ID":"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7","Type":"ContainerDied","Data":"236a45bbecdb97b8c9ab767753bc6e8ed506280be126a9fb0cf863f326e843a5"}
Oct 30 22:23:32.909836 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:32.909813    4871 scope.go:122] "RemoveContainer" containerID="236a45bbecdb97b8c9ab767753bc6e8ed506280be126a9fb0cf863f326e843a5"
Oct 30 22:23:32.911441 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:32.911414    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:23:32.911546 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:32.911484    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:23:32.911546 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:32.911521    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:23:33.749191 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:33.749150    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd"]
Oct 30 22:23:33.753074 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:33.753045    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd"
Oct 30 22:23:33.820296 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:33.816219    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-b-7fd427-serving-cert\""
Oct 30 22:23:33.866321 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:33.866285    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd"]
Oct 30 22:23:33.928386 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:33.928053    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38c81c78-6dca-4ec6-9347-bf512bcc0d05-metrics-certs\") pod \"maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd\" (UID: \"38c81c78-6dca-4ec6-9347-bf512bcc0d05\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd"
Oct 30 22:23:33.928386 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:33.928193    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38c81c78-6dca-4ec6-9347-bf512bcc0d05-maas-api-tls\") pod \"maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd\" (UID: \"38c81c78-6dca-4ec6-9347-bf512bcc0d05\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd"
Oct 30 22:23:33.928386 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:33.928279    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38c81c78-6dca-4ec6-9347-bf512bcc0d05-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd\" (UID: \"38c81c78-6dca-4ec6-9347-bf512bcc0d05\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd"
Oct 30 22:23:33.928386 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:33.928318    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4m5cr\" (UniqueName: \"kubernetes.io/projected/38c81c78-6dca-4ec6-9347-bf512bcc0d05-kube-api-access-4m5cr\") pod \"maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd\" (UID: \"38c81c78-6dca-4ec6-9347-bf512bcc0d05\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd"
Oct 30 22:23:33.961732 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:33.961698    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9" event={"ID":"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7","Type":"ContainerStarted","Data":"15458c6d8fcd1a410d6d8a7124ea8545ba3d15d34e29f452ca03a24b027c395f"}
Oct 30 22:23:33.967882 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:33.967853    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:34.061314 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:34.043635    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38c81c78-6dca-4ec6-9347-bf512bcc0d05-maas-api-tls\") pod \"maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd\" (UID: \"38c81c78-6dca-4ec6-9347-bf512bcc0d05\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd"
Oct 30 22:23:34.061314 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:34.043735    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38c81c78-6dca-4ec6-9347-bf512bcc0d05-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd\" (UID: \"38c81c78-6dca-4ec6-9347-bf512bcc0d05\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd"
Oct 30 22:23:34.061314 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:34.043793    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4m5cr\" (UniqueName: \"kubernetes.io/projected/38c81c78-6dca-4ec6-9347-bf512bcc0d05-kube-api-access-4m5cr\") pod \"maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd\" (UID: \"38c81c78-6dca-4ec6-9347-bf512bcc0d05\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd"
Oct 30 22:23:34.061314 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:34.043946    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38c81c78-6dca-4ec6-9347-bf512bcc0d05-metrics-certs\") pod \"maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd\" (UID: \"38c81c78-6dca-4ec6-9347-bf512bcc0d05\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd"
Oct 30 22:23:34.061314 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:34.045294    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38c81c78-6dca-4ec6-9347-bf512bcc0d05-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd\" (UID: \"38c81c78-6dca-4ec6-9347-bf512bcc0d05\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd"
Oct 30 22:23:34.061314 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:23:34.045398    4871 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-praxis-b-7fd427-serving-cert: secret "maas-api-e2e-praxis-b-7fd427-serving-cert" not found
Oct 30 22:23:34.061314 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:23:34.045457    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38c81c78-6dca-4ec6-9347-bf512bcc0d05-maas-api-tls podName:38c81c78-6dca-4ec6-9347-bf512bcc0d05 nodeName:}" failed. No retries permitted until 2026-10-30 22:23:34.545437479 +0000 UTC m=+2756.722926088 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/38c81c78-6dca-4ec6-9347-bf512bcc0d05-maas-api-tls") pod "maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd" (UID: "38c81c78-6dca-4ec6-9347-bf512bcc0d05") : secret "maas-api-e2e-praxis-b-7fd427-serving-cert" not found
Oct 30 22:23:34.079199 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:34.079052    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38c81c78-6dca-4ec6-9347-bf512bcc0d05-metrics-certs\") pod \"maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd\" (UID: \"38c81c78-6dca-4ec6-9347-bf512bcc0d05\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd"
Oct 30 22:23:34.123996 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:34.123935    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:34.137260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:34.136017    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4m5cr\" (UniqueName: \"kubernetes.io/projected/38c81c78-6dca-4ec6-9347-bf512bcc0d05-kube-api-access-4m5cr\") pod \"maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd\" (UID: \"38c81c78-6dca-4ec6-9347-bf512bcc0d05\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd"
Oct 30 22:23:34.167684 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:34.167653    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"
Oct 30 22:23:34.168080 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:34.168042    4871 patch_prober.go:29] interesting pod/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused" start-of-body=
Oct 30 22:23:34.168252 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:34.168215    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9" podUID="689c9d7f-c8ab-49e5-b647-b4a62d97c1e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused"
Oct 30 22:23:34.436505 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:34.436295    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:34.552916 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:34.552869    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38c81c78-6dca-4ec6-9347-bf512bcc0d05-maas-api-tls\") pod \"maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd\" (UID: \"38c81c78-6dca-4ec6-9347-bf512bcc0d05\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd"
Oct 30 22:23:34.555815 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:34.555778    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38c81c78-6dca-4ec6-9347-bf512bcc0d05-maas-api-tls\") pod \"maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd\" (UID: \"38c81c78-6dca-4ec6-9347-bf512bcc0d05\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd"
Oct 30 22:23:34.666680 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:34.666642    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd"
Oct 30 22:23:35.089186 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:35.089150    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd"]
Oct 30 22:23:35.090151 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:23:35.090125    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38c81c78_6dca_4ec6_9347_bf512bcc0d05.slice/crio-389e46bd71d8b8c029c6607570ef3526737eca7d25d0a3dce640d621633d5a43 WatchSource:0}: Error finding container 389e46bd71d8b8c029c6607570ef3526737eca7d25d0a3dce640d621633d5a43: Status 404 returned error can't find the container with id 389e46bd71d8b8c029c6607570ef3526737eca7d25d0a3dce640d621633d5a43
Oct 30 22:23:35.166576 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:35.166542    4871 patch_prober.go:29] interesting pod/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused" start-of-body=
Oct 30 22:23:35.166768 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:35.166595    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9" podUID="689c9d7f-c8ab-49e5-b647-b4a62d97c1e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused"
Oct 30 22:23:35.570033 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:23:35.569996    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:23:35.973578 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:35.973543    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd" event={"ID":"38c81c78-6dca-4ec6-9347-bf512bcc0d05","Type":"ContainerStarted","Data":"389e46bd71d8b8c029c6607570ef3526737eca7d25d0a3dce640d621633d5a43"}
Oct 30 22:23:36.166643 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:36.166615    4871 patch_prober.go:29] interesting pod/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused" start-of-body=
Oct 30 22:23:36.166928 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:36.166699    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9" podUID="689c9d7f-c8ab-49e5-b647-b4a62d97c1e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused"
Oct 30 22:23:36.987769 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:36.987732    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd" event={"ID":"38c81c78-6dca-4ec6-9347-bf512bcc0d05","Type":"ContainerStarted","Data":"c354fae03e416c897090f452786797e3c3b9ad2824eeadd3b7035297acd1b121"}
Oct 30 22:23:36.987959 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:36.987807    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd"
Oct 30 22:23:37.139514 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:37.138375    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd" podStartSLOduration=3.185796052 podStartE2EDuration="4.138352828s" totalImagesPullingTime="952.556776ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:23:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:23:37.138079618 +0000 UTC m=+2759.315568236" watchObservedRunningTime="2026-10-30 22:23:37.138352828 +0000 UTC m=+2759.315841446"
Oct 30 22:23:37.166248 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:37.166202    4871 patch_prober.go:29] interesting pod/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused" start-of-body=
Oct 30 22:23:37.166432 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:37.166277    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9" podUID="689c9d7f-c8ab-49e5-b647-b4a62d97c1e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused"
Oct 30 22:23:38.166421 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:38.166388    4871 patch_prober.go:29] interesting pod/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused" start-of-body=
Oct 30 22:23:38.166821 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:38.166444    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9" podUID="689c9d7f-c8ab-49e5-b647-b4a62d97c1e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused"
Oct 30 22:23:39.062764 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:39.062730    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:39.166984 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:39.166945    4871 patch_prober.go:29] interesting pod/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused" start-of-body=
Oct 30 22:23:39.167419 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:39.167009    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9" podUID="689c9d7f-c8ab-49e5-b647-b4a62d97c1e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused"
Oct 30 22:23:40.170604 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:40.170563    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"
Oct 30 22:23:40.171100 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:40.170987    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"
Oct 30 22:23:40.171779 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:40.171757    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"
Oct 30 22:23:42.430221 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:42.430188    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:42.591120 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:42.591091    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:43.020873 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:43.020843    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd"
Oct 30 22:23:44.007214 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:44.007177    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:44.416766 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:44.416735    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:50.102461 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:50.102427    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:54.647592 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:54.647526    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:59.309458 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:23:59.309423    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:02.978527 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:02.978491    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:04.452269 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:04.452197    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:05.570379 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:24:05.570342    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:24:07.216439 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:07.216383    4871 patch_prober.go:29] interesting pod/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.167:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:24:07.216979 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:07.216472    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz" podUID="f4aaa6ee-3ee7-44f4-93ab-673837d4a23b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.167:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:24:08.534960 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:08.534918    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx"]
Oct 30 22:24:08.535464 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:08.535298    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx" podUID="53120b48-0737-4956-965b-bf544e8c1096" containerName="maas-api" containerID="cri-o://0c1800f416abdf4483d07aafdf87668edee24edcf7f9a0c0dbf87d43ea510bf7" gracePeriod=30
Oct 30 22:24:08.643905 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:08.643854    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:08.646664 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:08.646630    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx" podUID="53120b48-0737-4956-965b-bf544e8c1096" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.161:8443/health\": dial tcp 10.133.0.161:8443: connect: connection refused"
Oct 30 22:24:08.884669 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:08.884637    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx"
Oct 30 22:24:09.006473 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:09.006386    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/53120b48-0737-4956-965b-bf544e8c1096-kube-api-access-4rqhh\" (UniqueName: \"kubernetes.io/projected/53120b48-0737-4956-965b-bf544e8c1096-kube-api-access-4rqhh\") pod \"53120b48-0737-4956-965b-bf544e8c1096\" (UID: \"53120b48-0737-4956-965b-bf544e8c1096\") "
Oct 30 22:24:09.006473 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:09.006454    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/53120b48-0737-4956-965b-bf544e8c1096-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53120b48-0737-4956-965b-bf544e8c1096-openshift-service-ca\") pod \"53120b48-0737-4956-965b-bf544e8c1096\" (UID: \"53120b48-0737-4956-965b-bf544e8c1096\") "
Oct 30 22:24:09.006754 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:09.006641    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/53120b48-0737-4956-965b-bf544e8c1096-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53120b48-0737-4956-965b-bf544e8c1096-maas-api-tls\") pod \"53120b48-0737-4956-965b-bf544e8c1096\" (UID: \"53120b48-0737-4956-965b-bf544e8c1096\") "
Oct 30 22:24:09.006821 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:09.006751    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/53120b48-0737-4956-965b-bf544e8c1096-metrics-certs\" (UniqueName: \"kubernetes.io/secret/53120b48-0737-4956-965b-bf544e8c1096-metrics-certs\") pod \"53120b48-0737-4956-965b-bf544e8c1096\" (UID: \"53120b48-0737-4956-965b-bf544e8c1096\") "
Oct 30 22:24:09.006959 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:09.006907    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53120b48-0737-4956-965b-bf544e8c1096-openshift-service-ca" pod "53120b48-0737-4956-965b-bf544e8c1096" (UID: "53120b48-0737-4956-965b-bf544e8c1096"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:24:09.007414 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:09.007389    4871 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53120b48-0737-4956-965b-bf544e8c1096-openshift-service-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:09.009508 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:09.009460    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53120b48-0737-4956-965b-bf544e8c1096-metrics-certs" pod "53120b48-0737-4956-965b-bf544e8c1096" (UID: "53120b48-0737-4956-965b-bf544e8c1096"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:24:09.009633 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:09.009513    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53120b48-0737-4956-965b-bf544e8c1096-kube-api-access-4rqhh" pod "53120b48-0737-4956-965b-bf544e8c1096" (UID: "53120b48-0737-4956-965b-bf544e8c1096"). InnerVolumeSpecName "kube-api-access-4rqhh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:24:09.009695 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:09.009662    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53120b48-0737-4956-965b-bf544e8c1096-maas-api-tls" pod "53120b48-0737-4956-965b-bf544e8c1096" (UID: "53120b48-0737-4956-965b-bf544e8c1096"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:24:09.108536 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:09.108437    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rqhh\" (UniqueName: \"kubernetes.io/projected/53120b48-0737-4956-965b-bf544e8c1096-kube-api-access-4rqhh\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:09.108536 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:09.108474    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53120b48-0737-4956-965b-bf544e8c1096-maas-api-tls\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:09.108536 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:09.108485    4871 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53120b48-0737-4956-965b-bf544e8c1096-metrics-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:09.300027 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:09.299991    4871 generic.go:356] "Generic (PLEG): container finished" podID="53120b48-0737-4956-965b-bf544e8c1096" containerID="0c1800f416abdf4483d07aafdf87668edee24edcf7f9a0c0dbf87d43ea510bf7" exitCode=0
Oct 30 22:24:09.300211 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:09.300048    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx" event={"ID":"53120b48-0737-4956-965b-bf544e8c1096","Type":"ContainerDied","Data":"0c1800f416abdf4483d07aafdf87668edee24edcf7f9a0c0dbf87d43ea510bf7"}
Oct 30 22:24:09.300211 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:09.300056    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx"
Oct 30 22:24:09.300211 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:09.300091    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx" event={"ID":"53120b48-0737-4956-965b-bf544e8c1096","Type":"ContainerDied","Data":"fbe0d7bda18ac5a2f375a6be0be2eb5d874b8195575b69d2a39793cbe3d9f516"}
Oct 30 22:24:09.300211 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:09.300114    4871 scope.go:122] "RemoveContainer" containerID="0c1800f416abdf4483d07aafdf87668edee24edcf7f9a0c0dbf87d43ea510bf7"
Oct 30 22:24:09.313352 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:09.313325    4871 scope.go:122] "RemoveContainer" containerID="0c1800f416abdf4483d07aafdf87668edee24edcf7f9a0c0dbf87d43ea510bf7"
Oct 30 22:24:09.313749 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:24:09.313724    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c1800f416abdf4483d07aafdf87668edee24edcf7f9a0c0dbf87d43ea510bf7\": container with ID starting with 0c1800f416abdf4483d07aafdf87668edee24edcf7f9a0c0dbf87d43ea510bf7 not found: ID does not exist" containerID="0c1800f416abdf4483d07aafdf87668edee24edcf7f9a0c0dbf87d43ea510bf7"
Oct 30 22:24:09.313811 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:09.313759    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c1800f416abdf4483d07aafdf87668edee24edcf7f9a0c0dbf87d43ea510bf7"} err="failed to get container status \"0c1800f416abdf4483d07aafdf87668edee24edcf7f9a0c0dbf87d43ea510bf7\": rpc error: code = NotFound desc = could not find container \"0c1800f416abdf4483d07aafdf87668edee24edcf7f9a0c0dbf87d43ea510bf7\": container with ID starting with 0c1800f416abdf4483d07aafdf87668edee24edcf7f9a0c0dbf87d43ea510bf7 not found: ID does not exist"
Oct 30 22:24:09.460892 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:09.460846    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx"]
Oct 30 22:24:09.466790 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:09.465395    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-3cecc7-7bb6b45c54-sq9bx"]
Oct 30 22:24:09.961580 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:09.961495    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="53120b48-0737-4956-965b-bf544e8c1096" path="/var/lib/kubelet/pods/53120b48-0737-4956-965b-bf544e8c1096/volumes"
Oct 30 22:24:10.279018 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:10.278985    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:10.310304 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:10.310266    4871 generic.go:356] "Generic (PLEG): container finished" podID="f4aaa6ee-3ee7-44f4-93ab-673837d4a23b" containerID="446a7c884a4280e9cfbcd0dfc303c3f87bafa507bc5c657fde1ecfeba6339ab8" exitCode=0
Oct 30 22:24:10.310489 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:10.310336    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz" event={"ID":"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b","Type":"ContainerDied","Data":"446a7c884a4280e9cfbcd0dfc303c3f87bafa507bc5c657fde1ecfeba6339ab8"}
Oct 30 22:24:10.310818 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:10.310801    4871 scope.go:122] "RemoveContainer" containerID="446a7c884a4280e9cfbcd0dfc303c3f87bafa507bc5c657fde1ecfeba6339ab8"
Oct 30 22:24:10.312594 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:10.312562    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:24:10.312734 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:10.312652    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:24:10.312734 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:10.312723    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:24:10.494061 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:10.494003    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:10.650609 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:10.650577    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:11.335544 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:11.335508    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz" event={"ID":"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b","Type":"ContainerStarted","Data":"b6d3d7c5aa4164e9e468b3141618f001cd02ffca5fd5df0b287d5ccdac19152b"}
Oct 30 22:24:11.424210 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:11.424172    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:11.895641 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:11.895608    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"
Oct 30 22:24:11.895886 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:11.895861    4871 patch_prober.go:29] interesting pod/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused" start-of-body=
Oct 30 22:24:11.895986 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:11.895921    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz" podUID="f4aaa6ee-3ee7-44f4-93ab-673837d4a23b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused"
Oct 30 22:24:12.895435 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:12.895388    4871 patch_prober.go:29] interesting pod/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused" start-of-body=
Oct 30 22:24:12.895960 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:12.895460    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz" podUID="f4aaa6ee-3ee7-44f4-93ab-673837d4a23b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused"
Oct 30 22:24:13.896016 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:13.895977    4871 patch_prober.go:29] interesting pod/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused" start-of-body=
Oct 30 22:24:13.896465 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:13.896048    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz" podUID="f4aaa6ee-3ee7-44f4-93ab-673837d4a23b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused"
Oct 30 22:24:14.896015 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:14.895975    4871 patch_prober.go:29] interesting pod/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused" start-of-body=
Oct 30 22:24:14.896217 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:14.896035    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz" podUID="f4aaa6ee-3ee7-44f4-93ab-673837d4a23b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused"
Oct 30 22:24:15.506547 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:15.506506    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:15.895532 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:15.895491    4871 patch_prober.go:29] interesting pod/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused" start-of-body=
Oct 30 22:24:15.895718 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:15.895550    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz" podUID="f4aaa6ee-3ee7-44f4-93ab-673837d4a23b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused"
Oct 30 22:24:16.899577 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:16.899539    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"
Oct 30 22:24:16.900108 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:16.899928    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"
Oct 30 22:24:16.900678 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:16.900651    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"
Oct 30 22:24:17.262758 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:17.262665    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:17.462064 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:17.462033    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:17.594297 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:17.594187    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"]
Oct 30 22:24:17.594496 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:17.594435    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb" podUID="1fad116b-e1cc-42da-a439-bdecb46b96a9" containerName="istio-proxy" containerID="cri-o://41cec9c0dfdcd0be3f7cdeebbcafa2f7bbc1a8d31c75a8878ff8fb9d94734030" gracePeriod=30
Oct 30 22:24:17.634979 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:17.634944    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:20.330629 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:20.330582    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:22.625165 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:22.625135    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:22.830759 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:22.830606    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:22.912937 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:22.912911    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"
Oct 30 22:24:23.017577 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:23.017542    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1fad116b-e1cc-42da-a439-bdecb46b96a9-istio-token\" (UniqueName: \"kubernetes.io/projected/1fad116b-e1cc-42da-a439-bdecb46b96a9-istio-token\") pod \"1fad116b-e1cc-42da-a439-bdecb46b96a9\" (UID: \"1fad116b-e1cc-42da-a439-bdecb46b96a9\") "
Oct 30 22:24:23.017800 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:23.017605    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1fad116b-e1cc-42da-a439-bdecb46b96a9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fad116b-e1cc-42da-a439-bdecb46b96a9-istio-podinfo\") pod \"1fad116b-e1cc-42da-a439-bdecb46b96a9\" (UID: \"1fad116b-e1cc-42da-a439-bdecb46b96a9\") "
Oct 30 22:24:23.017800 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:23.017645    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1fad116b-e1cc-42da-a439-bdecb46b96a9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fad116b-e1cc-42da-a439-bdecb46b96a9-istio-envoy\") pod \"1fad116b-e1cc-42da-a439-bdecb46b96a9\" (UID: \"1fad116b-e1cc-42da-a439-bdecb46b96a9\") "
Oct 30 22:24:23.018081 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:23.018054    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1fad116b-e1cc-42da-a439-bdecb46b96a9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fad116b-e1cc-42da-a439-bdecb46b96a9-istio-data\") pod \"1fad116b-e1cc-42da-a439-bdecb46b96a9\" (UID: \"1fad116b-e1cc-42da-a439-bdecb46b96a9\") "
Oct 30 22:24:23.018183 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:23.018113    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1fad116b-e1cc-42da-a439-bdecb46b96a9-kube-api-access-5sdhr\" (UniqueName: \"kubernetes.io/projected/1fad116b-e1cc-42da-a439-bdecb46b96a9-kube-api-access-5sdhr\") pod \"1fad116b-e1cc-42da-a439-bdecb46b96a9\" (UID: \"1fad116b-e1cc-42da-a439-bdecb46b96a9\") "
Oct 30 22:24:23.018183 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:23.018150    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1fad116b-e1cc-42da-a439-bdecb46b96a9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fad116b-e1cc-42da-a439-bdecb46b96a9-workload-certs\") pod \"1fad116b-e1cc-42da-a439-bdecb46b96a9\" (UID: \"1fad116b-e1cc-42da-a439-bdecb46b96a9\") "
Oct 30 22:24:23.018334 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:23.018273    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1fad116b-e1cc-42da-a439-bdecb46b96a9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fad116b-e1cc-42da-a439-bdecb46b96a9-workload-socket\") pod \"1fad116b-e1cc-42da-a439-bdecb46b96a9\" (UID: \"1fad116b-e1cc-42da-a439-bdecb46b96a9\") "
Oct 30 22:24:23.018391 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:23.018349    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1fad116b-e1cc-42da-a439-bdecb46b96a9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fad116b-e1cc-42da-a439-bdecb46b96a9-istiod-ca-cert\") pod \"1fad116b-e1cc-42da-a439-bdecb46b96a9\" (UID: \"1fad116b-e1cc-42da-a439-bdecb46b96a9\") "
Oct 30 22:24:23.018433 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:23.018406    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1fad116b-e1cc-42da-a439-bdecb46b96a9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fad116b-e1cc-42da-a439-bdecb46b96a9-credential-socket\") pod \"1fad116b-e1cc-42da-a439-bdecb46b96a9\" (UID: \"1fad116b-e1cc-42da-a439-bdecb46b96a9\") "
Oct 30 22:24:23.018741 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:23.018665    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fad116b-e1cc-42da-a439-bdecb46b96a9-workload-certs" pod "1fad116b-e1cc-42da-a439-bdecb46b96a9" (UID: "1fad116b-e1cc-42da-a439-bdecb46b96a9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:24:23.018893 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:23.018801    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fad116b-e1cc-42da-a439-bdecb46b96a9-istio-data" pod "1fad116b-e1cc-42da-a439-bdecb46b96a9" (UID: "1fad116b-e1cc-42da-a439-bdecb46b96a9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:24:23.018981 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:23.018962    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fad116b-e1cc-42da-a439-bdecb46b96a9-credential-socket" pod "1fad116b-e1cc-42da-a439-bdecb46b96a9" (UID: "1fad116b-e1cc-42da-a439-bdecb46b96a9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:24:23.019079 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:23.019055    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fad116b-e1cc-42da-a439-bdecb46b96a9-istiod-ca-cert" pod "1fad116b-e1cc-42da-a439-bdecb46b96a9" (UID: "1fad116b-e1cc-42da-a439-bdecb46b96a9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:24:23.019302 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:23.019282    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fad116b-e1cc-42da-a439-bdecb46b96a9-workload-socket" pod "1fad116b-e1cc-42da-a439-bdecb46b96a9" (UID: "1fad116b-e1cc-42da-a439-bdecb46b96a9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:24:23.020069 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:23.020040    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fad116b-e1cc-42da-a439-bdecb46b96a9-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:23.020069 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:23.020062    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fad116b-e1cc-42da-a439-bdecb46b96a9-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:23.020394 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:23.020077    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fad116b-e1cc-42da-a439-bdecb46b96a9-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:23.020394 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:23.020093    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fad116b-e1cc-42da-a439-bdecb46b96a9-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:23.020394 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:23.020107    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fad116b-e1cc-42da-a439-bdecb46b96a9-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:23.020394 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:23.020332    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1fad116b-e1cc-42da-a439-bdecb46b96a9-istio-podinfo" pod "1fad116b-e1cc-42da-a439-bdecb46b96a9" (UID: "1fad116b-e1cc-42da-a439-bdecb46b96a9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:24:23.020394 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:23.020383    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fad116b-e1cc-42da-a439-bdecb46b96a9-istio-token" pod "1fad116b-e1cc-42da-a439-bdecb46b96a9" (UID: "1fad116b-e1cc-42da-a439-bdecb46b96a9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:24:23.020763 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:23.020738    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fad116b-e1cc-42da-a439-bdecb46b96a9-istio-envoy" pod "1fad116b-e1cc-42da-a439-bdecb46b96a9" (UID: "1fad116b-e1cc-42da-a439-bdecb46b96a9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:24:23.021606 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:23.021585    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fad116b-e1cc-42da-a439-bdecb46b96a9-kube-api-access-5sdhr" pod "1fad116b-e1cc-42da-a439-bdecb46b96a9" (UID: "1fad116b-e1cc-42da-a439-bdecb46b96a9"). InnerVolumeSpecName "kube-api-access-5sdhr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:24:23.121172 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:23.121131    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fad116b-e1cc-42da-a439-bdecb46b96a9-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:23.121172 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:23.121172    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fad116b-e1cc-42da-a439-bdecb46b96a9-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:23.121436 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:23.121190    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fad116b-e1cc-42da-a439-bdecb46b96a9-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:23.121436 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:23.121204    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5sdhr\" (UniqueName: \"kubernetes.io/projected/1fad116b-e1cc-42da-a439-bdecb46b96a9-kube-api-access-5sdhr\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:23.441148 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:23.441111    4871 generic.go:356] "Generic (PLEG): container finished" podID="1fad116b-e1cc-42da-a439-bdecb46b96a9" containerID="41cec9c0dfdcd0be3f7cdeebbcafa2f7bbc1a8d31c75a8878ff8fb9d94734030" exitCode=0
Oct 30 22:24:23.441362 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:23.441179    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"
Oct 30 22:24:23.441362 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:23.441197    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb" event={"ID":"1fad116b-e1cc-42da-a439-bdecb46b96a9","Type":"ContainerDied","Data":"41cec9c0dfdcd0be3f7cdeebbcafa2f7bbc1a8d31c75a8878ff8fb9d94734030"}
Oct 30 22:24:23.441362 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:23.441322    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb" event={"ID":"1fad116b-e1cc-42da-a439-bdecb46b96a9","Type":"ContainerDied","Data":"1e5b5fb524e298a03474138035bc46c19a76048153e7436c74e31980f271f959"}
Oct 30 22:24:23.441362 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:23.441342    4871 scope.go:122] "RemoveContainer" containerID="41cec9c0dfdcd0be3f7cdeebbcafa2f7bbc1a8d31c75a8878ff8fb9d94734030"
Oct 30 22:24:23.453884 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:23.453857    4871 scope.go:122] "RemoveContainer" containerID="6925013eb32b2bc795b436ed113177007fa8629b8f2b226802e10ab405000376"
Oct 30 22:24:23.464718 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:23.464695    4871 scope.go:122] "RemoveContainer" containerID="41cec9c0dfdcd0be3f7cdeebbcafa2f7bbc1a8d31c75a8878ff8fb9d94734030"
Oct 30 22:24:23.465066 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:24:23.465044    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41cec9c0dfdcd0be3f7cdeebbcafa2f7bbc1a8d31c75a8878ff8fb9d94734030\": container with ID starting with 41cec9c0dfdcd0be3f7cdeebbcafa2f7bbc1a8d31c75a8878ff8fb9d94734030 not found: ID does not exist" containerID="41cec9c0dfdcd0be3f7cdeebbcafa2f7bbc1a8d31c75a8878ff8fb9d94734030"
Oct 30 22:24:23.465115 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:23.465077    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41cec9c0dfdcd0be3f7cdeebbcafa2f7bbc1a8d31c75a8878ff8fb9d94734030"} err="failed to get container status \"41cec9c0dfdcd0be3f7cdeebbcafa2f7bbc1a8d31c75a8878ff8fb9d94734030\": rpc error: code = NotFound desc = could not find container \"41cec9c0dfdcd0be3f7cdeebbcafa2f7bbc1a8d31c75a8878ff8fb9d94734030\": container with ID starting with 41cec9c0dfdcd0be3f7cdeebbcafa2f7bbc1a8d31c75a8878ff8fb9d94734030 not found: ID does not exist"
Oct 30 22:24:23.465115 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:23.465095    4871 scope.go:122] "RemoveContainer" containerID="6925013eb32b2bc795b436ed113177007fa8629b8f2b226802e10ab405000376"
Oct 30 22:24:23.465338 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:24:23.465321    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6925013eb32b2bc795b436ed113177007fa8629b8f2b226802e10ab405000376\": container with ID starting with 6925013eb32b2bc795b436ed113177007fa8629b8f2b226802e10ab405000376 not found: ID does not exist" containerID="6925013eb32b2bc795b436ed113177007fa8629b8f2b226802e10ab405000376"
Oct 30 22:24:23.465394 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:23.465343    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6925013eb32b2bc795b436ed113177007fa8629b8f2b226802e10ab405000376"} err="failed to get container status \"6925013eb32b2bc795b436ed113177007fa8629b8f2b226802e10ab405000376\": rpc error: code = NotFound desc = could not find container \"6925013eb32b2bc795b436ed113177007fa8629b8f2b226802e10ab405000376\": container with ID starting with 6925013eb32b2bc795b436ed113177007fa8629b8f2b226802e10ab405000376 not found: ID does not exist"
Oct 30 22:24:23.610867 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:23.605177    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"]
Oct 30 22:24:23.649557 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:23.649492    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-a-3cecc7-openshift-default-668489bb6f-4vplb"]
Oct 30 22:24:23.979204 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:23.979156    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1fad116b-e1cc-42da-a439-bdecb46b96a9" path="/var/lib/kubelet/pods/1fad116b-e1cc-42da-a439-bdecb46b96a9/volumes"
Oct 30 22:24:25.176288 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:25.176250    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:29.245748 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:29.245712    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:29.775414 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:29.775385    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:29.926767 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:29.926722    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:31.104754 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:31.104714    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:34.139924 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:34.139870    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:34.392258 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:34.392155    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:35.042202 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:35.042161    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:35.568699 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:24:35.568664    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:24:35.755530 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:35.755495    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:37.699661 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:37.699624    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:40.992582 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:40.992534    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:41.155594 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:41.155552    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:41.327762 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:41.327724    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:41.486201 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:41.486086    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:41.666858 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:41.666823    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"]
Oct 30 22:24:41.667099 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:41.667074    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg" podUID="dd4e1319-f9e3-46e5-856b-67e2f5139b6e" containerName="istio-proxy" containerID="cri-o://24cc7199824b1069243be9669a8ba55fa9058ed60035991411cfa5789f999b73" gracePeriod=30
Oct 30 22:24:41.693024 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:41.692990    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:42.791320 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:42.791281    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:43.055147 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:43.055056    4871 patch_prober.go:29] interesting pod/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:24:43.055359 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:43.055136    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg" podUID="dd4e1319-f9e3-46e5-856b-67e2f5139b6e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:24:44.757375 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:44.757334    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:46.951371 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:46.951293    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"
Oct 30 22:24:47.053322 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.053283    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-istiod-ca-cert\") pod \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\" (UID: \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\") "
Oct 30 22:24:47.053556 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.053366    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-istio-podinfo\") pod \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\" (UID: \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\") "
Oct 30 22:24:47.053556 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.053401    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-workload-socket\") pod \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\" (UID: \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\") "
Oct 30 22:24:47.053556 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.053430    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-istio-token\" (UniqueName: \"kubernetes.io/projected/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-istio-token\") pod \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\" (UID: \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\") "
Oct 30 22:24:47.053556 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.053465    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-credential-socket\") pod \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\" (UID: \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\") "
Oct 30 22:24:47.053800 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.053631    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-kube-api-access-fgqll\" (UniqueName: \"kubernetes.io/projected/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-kube-api-access-fgqll\") pod \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\" (UID: \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\") "
Oct 30 22:24:47.053800 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.053694    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-istio-data\") pod \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\" (UID: \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\") "
Oct 30 22:24:47.053800 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.053785    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-istio-envoy\") pod \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\" (UID: \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\") "
Oct 30 22:24:47.053981 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.053831    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-credential-socket" pod "dd4e1319-f9e3-46e5-856b-67e2f5139b6e" (UID: "dd4e1319-f9e3-46e5-856b-67e2f5139b6e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:24:47.053981 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.053905    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-workload-certs\") pod \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\" (UID: \"dd4e1319-f9e3-46e5-856b-67e2f5139b6e\") "
Oct 30 22:24:47.053981 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.053937    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-istiod-ca-cert" pod "dd4e1319-f9e3-46e5-856b-67e2f5139b6e" (UID: "dd4e1319-f9e3-46e5-856b-67e2f5139b6e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:24:47.053981 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.053912    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-workload-socket" pod "dd4e1319-f9e3-46e5-856b-67e2f5139b6e" (UID: "dd4e1319-f9e3-46e5-856b-67e2f5139b6e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:24:47.054169 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.054011    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-istio-data" pod "dd4e1319-f9e3-46e5-856b-67e2f5139b6e" (UID: "dd4e1319-f9e3-46e5-856b-67e2f5139b6e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:24:47.054291 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.054269    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-workload-certs" pod "dd4e1319-f9e3-46e5-856b-67e2f5139b6e" (UID: "dd4e1319-f9e3-46e5-856b-67e2f5139b6e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:24:47.054454 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.054435    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:47.054454 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.054455    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:47.054612 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.054464    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:47.054612 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.054474    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:47.054612 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.054483    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:47.055946 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.055913    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-istio-token" pod "dd4e1319-f9e3-46e5-856b-67e2f5139b6e" (UID: "dd4e1319-f9e3-46e5-856b-67e2f5139b6e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:24:47.056256 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.056223    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-istio-podinfo" pod "dd4e1319-f9e3-46e5-856b-67e2f5139b6e" (UID: "dd4e1319-f9e3-46e5-856b-67e2f5139b6e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:24:47.056406 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.056389    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-istio-envoy" pod "dd4e1319-f9e3-46e5-856b-67e2f5139b6e" (UID: "dd4e1319-f9e3-46e5-856b-67e2f5139b6e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:24:47.056450 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.056400    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-kube-api-access-fgqll" pod "dd4e1319-f9e3-46e5-856b-67e2f5139b6e" (UID: "dd4e1319-f9e3-46e5-856b-67e2f5139b6e"). InnerVolumeSpecName "kube-api-access-fgqll". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:24:47.156198 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.156158    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:47.156198 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.156199    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:47.156462 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.156214    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fgqll\" (UniqueName: \"kubernetes.io/projected/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-kube-api-access-fgqll\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:47.156462 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.156230    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd4e1319-f9e3-46e5-856b-67e2f5139b6e-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:47.442498 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.442463    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:47.650019 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.649591    4871 generic.go:356] "Generic (PLEG): container finished" podID="dd4e1319-f9e3-46e5-856b-67e2f5139b6e" containerID="24cc7199824b1069243be9669a8ba55fa9058ed60035991411cfa5789f999b73" exitCode=0
Oct 30 22:24:47.650019 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.649776    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg" event={"ID":"dd4e1319-f9e3-46e5-856b-67e2f5139b6e","Type":"ContainerDied","Data":"24cc7199824b1069243be9669a8ba55fa9058ed60035991411cfa5789f999b73"}
Oct 30 22:24:47.650019 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.649829    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg" event={"ID":"dd4e1319-f9e3-46e5-856b-67e2f5139b6e","Type":"ContainerDied","Data":"c7f0b9773e814c0c443c4b045966280db8d963624f1d780f2a0872d69268e3f5"}
Oct 30 22:24:47.650019 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.649858    4871 scope.go:122] "RemoveContainer" containerID="24cc7199824b1069243be9669a8ba55fa9058ed60035991411cfa5789f999b73"
Oct 30 22:24:47.650439 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.650043    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"
Oct 30 22:24:47.666284 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.666258    4871 scope.go:122] "RemoveContainer" containerID="bc6288e15eeee83ffb3730f14e83e7848119d8cc2485c97cb77a8b37d041b781"
Oct 30 22:24:47.676431 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.676403    4871 scope.go:122] "RemoveContainer" containerID="24cc7199824b1069243be9669a8ba55fa9058ed60035991411cfa5789f999b73"
Oct 30 22:24:47.676745 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:24:47.676723    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24cc7199824b1069243be9669a8ba55fa9058ed60035991411cfa5789f999b73\": container with ID starting with 24cc7199824b1069243be9669a8ba55fa9058ed60035991411cfa5789f999b73 not found: ID does not exist" containerID="24cc7199824b1069243be9669a8ba55fa9058ed60035991411cfa5789f999b73"
Oct 30 22:24:47.676817 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.676753    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24cc7199824b1069243be9669a8ba55fa9058ed60035991411cfa5789f999b73"} err="failed to get container status \"24cc7199824b1069243be9669a8ba55fa9058ed60035991411cfa5789f999b73\": rpc error: code = NotFound desc = could not find container \"24cc7199824b1069243be9669a8ba55fa9058ed60035991411cfa5789f999b73\": container with ID starting with 24cc7199824b1069243be9669a8ba55fa9058ed60035991411cfa5789f999b73 not found: ID does not exist"
Oct 30 22:24:47.676817 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.676773    4871 scope.go:122] "RemoveContainer" containerID="bc6288e15eeee83ffb3730f14e83e7848119d8cc2485c97cb77a8b37d041b781"
Oct 30 22:24:47.677059 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:24:47.677039    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc6288e15eeee83ffb3730f14e83e7848119d8cc2485c97cb77a8b37d041b781\": container with ID starting with bc6288e15eeee83ffb3730f14e83e7848119d8cc2485c97cb77a8b37d041b781 not found: ID does not exist" containerID="bc6288e15eeee83ffb3730f14e83e7848119d8cc2485c97cb77a8b37d041b781"
Oct 30 22:24:47.677112 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.677064    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc6288e15eeee83ffb3730f14e83e7848119d8cc2485c97cb77a8b37d041b781"} err="failed to get container status \"bc6288e15eeee83ffb3730f14e83e7848119d8cc2485c97cb77a8b37d041b781\": rpc error: code = NotFound desc = could not find container \"bc6288e15eeee83ffb3730f14e83e7848119d8cc2485c97cb77a8b37d041b781\": container with ID starting with bc6288e15eeee83ffb3730f14e83e7848119d8cc2485c97cb77a8b37d041b781 not found: ID does not exist"
Oct 30 22:24:47.808628 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.808527    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"]
Oct 30 22:24:47.855156 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.855120    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-b-85d00c-openshift-default-7699c795d9-rmdxg"]
Oct 30 22:24:47.960791 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:47.960717    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dd4e1319-f9e3-46e5-856b-67e2f5139b6e" path="/var/lib/kubelet/pods/dd4e1319-f9e3-46e5-856b-67e2f5139b6e/volumes"
Oct 30 22:24:51.193959 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:51.193924    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:51.358268 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:51.358151    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:51.548707 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:51.548619    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:53.075123 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:53.075087    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:53.423379 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:53.423336    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:53.596952 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:53.596903    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:54.058229 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:54.058196    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:56.761953 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:56.761903    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:59.347861 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:59.347821    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8"]
Oct 30 22:24:59.348327 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:59.348021    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8" podUID="c2b31083-b29f-4a68-a20c-853287a23d6e" containerName="maas-api" containerID="cri-o://26ed7cc26b7706f493abe24c9c0a9f4527309fe588591048ddf4ab4c47d79faa" gracePeriod=30
Oct 30 22:24:59.536353 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:59.536314    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:59.662068 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:59.662042    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8"
Oct 30 22:24:59.748272 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:59.748214    4871 generic.go:356] "Generic (PLEG): container finished" podID="c2b31083-b29f-4a68-a20c-853287a23d6e" containerID="26ed7cc26b7706f493abe24c9c0a9f4527309fe588591048ddf4ab4c47d79faa" exitCode=0
Oct 30 22:24:59.748477 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:59.748282    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8"
Oct 30 22:24:59.748477 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:59.748297    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8" event={"ID":"c2b31083-b29f-4a68-a20c-853287a23d6e","Type":"ContainerDied","Data":"26ed7cc26b7706f493abe24c9c0a9f4527309fe588591048ddf4ab4c47d79faa"}
Oct 30 22:24:59.748477 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:59.748363    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8" event={"ID":"c2b31083-b29f-4a68-a20c-853287a23d6e","Type":"ContainerDied","Data":"25388e45e969435b7749327111eca239786650c7578315c957ce2195c7b67a47"}
Oct 30 22:24:59.748477 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:59.748387    4871 scope.go:122] "RemoveContainer" containerID="26ed7cc26b7706f493abe24c9c0a9f4527309fe588591048ddf4ab4c47d79faa"
Oct 30 22:24:59.761720 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:59.761693    4871 scope.go:122] "RemoveContainer" containerID="26ed7cc26b7706f493abe24c9c0a9f4527309fe588591048ddf4ab4c47d79faa"
Oct 30 22:24:59.762933 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:24:59.762905    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26ed7cc26b7706f493abe24c9c0a9f4527309fe588591048ddf4ab4c47d79faa\": container with ID starting with 26ed7cc26b7706f493abe24c9c0a9f4527309fe588591048ddf4ab4c47d79faa not found: ID does not exist" containerID="26ed7cc26b7706f493abe24c9c0a9f4527309fe588591048ddf4ab4c47d79faa"
Oct 30 22:24:59.763065 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:59.762942    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26ed7cc26b7706f493abe24c9c0a9f4527309fe588591048ddf4ab4c47d79faa"} err="failed to get container status \"26ed7cc26b7706f493abe24c9c0a9f4527309fe588591048ddf4ab4c47d79faa\": rpc error: code = NotFound desc = could not find container \"26ed7cc26b7706f493abe24c9c0a9f4527309fe588591048ddf4ab4c47d79faa\": container with ID starting with 26ed7cc26b7706f493abe24c9c0a9f4527309fe588591048ddf4ab4c47d79faa not found: ID does not exist"
Oct 30 22:24:59.811332 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:59.811298    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c2b31083-b29f-4a68-a20c-853287a23d6e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2b31083-b29f-4a68-a20c-853287a23d6e-maas-api-tls\") pod \"c2b31083-b29f-4a68-a20c-853287a23d6e\" (UID: \"c2b31083-b29f-4a68-a20c-853287a23d6e\") "
Oct 30 22:24:59.811509 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:59.811350    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c2b31083-b29f-4a68-a20c-853287a23d6e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2b31083-b29f-4a68-a20c-853287a23d6e-openshift-service-ca\") pod \"c2b31083-b29f-4a68-a20c-853287a23d6e\" (UID: \"c2b31083-b29f-4a68-a20c-853287a23d6e\") "
Oct 30 22:24:59.811509 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:59.811437    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c2b31083-b29f-4a68-a20c-853287a23d6e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2b31083-b29f-4a68-a20c-853287a23d6e-metrics-certs\") pod \"c2b31083-b29f-4a68-a20c-853287a23d6e\" (UID: \"c2b31083-b29f-4a68-a20c-853287a23d6e\") "
Oct 30 22:24:59.811509 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:59.811492    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c2b31083-b29f-4a68-a20c-853287a23d6e-kube-api-access-wdz72\" (UniqueName: \"kubernetes.io/projected/c2b31083-b29f-4a68-a20c-853287a23d6e-kube-api-access-wdz72\") pod \"c2b31083-b29f-4a68-a20c-853287a23d6e\" (UID: \"c2b31083-b29f-4a68-a20c-853287a23d6e\") "
Oct 30 22:24:59.812094 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:59.812063    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2b31083-b29f-4a68-a20c-853287a23d6e-openshift-service-ca" pod "c2b31083-b29f-4a68-a20c-853287a23d6e" (UID: "c2b31083-b29f-4a68-a20c-853287a23d6e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:24:59.813887 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:59.813861    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2b31083-b29f-4a68-a20c-853287a23d6e-kube-api-access-wdz72" pod "c2b31083-b29f-4a68-a20c-853287a23d6e" (UID: "c2b31083-b29f-4a68-a20c-853287a23d6e"). InnerVolumeSpecName "kube-api-access-wdz72". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:24:59.814026 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:59.814011    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2b31083-b29f-4a68-a20c-853287a23d6e-maas-api-tls" pod "c2b31083-b29f-4a68-a20c-853287a23d6e" (UID: "c2b31083-b29f-4a68-a20c-853287a23d6e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:24:59.814272 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:59.814252    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2b31083-b29f-4a68-a20c-853287a23d6e-metrics-certs" pod "c2b31083-b29f-4a68-a20c-853287a23d6e" (UID: "c2b31083-b29f-4a68-a20c-853287a23d6e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:24:59.941327 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:59.922360    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wdz72\" (UniqueName: \"kubernetes.io/projected/c2b31083-b29f-4a68-a20c-853287a23d6e-kube-api-access-wdz72\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:59.941327 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:59.922401    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2b31083-b29f-4a68-a20c-853287a23d6e-maas-api-tls\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:59.941327 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:59.922417    4871 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2b31083-b29f-4a68-a20c-853287a23d6e-openshift-service-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:59.941327 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:24:59.922431    4871 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2b31083-b29f-4a68-a20c-853287a23d6e-metrics-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:00.050828 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:00.050794    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:00.172211 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:00.172173    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8"]
Oct 30 22:25:00.176441 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:00.176410    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-f4a424-557cc6c4f9-k7wk8"]
Oct 30 22:25:00.676416 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:00.676377    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:00.835923 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:00.835866    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:01.021465 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:01.021362    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:01.024734 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:01.024695    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"]
Oct 30 22:25:01.024914 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:01.024891    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g" podUID="4d334339-1d4b-421b-856e-c2a0d9f4737a" containerName="istio-proxy" containerID="cri-o://849ada11e535fa4bffd8d26586363c6193392257e1aab2a58b45119ea3f615de" gracePeriod=30
Oct 30 22:25:01.212259 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:01.198902    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:01.554965 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:01.554927    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:01.938090 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:01.938039    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:01.960850 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:01.960813    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c2b31083-b29f-4a68-a20c-853287a23d6e" path="/var/lib/kubelet/pods/c2b31083-b29f-4a68-a20c-853287a23d6e/volumes"
Oct 30 22:25:05.568472 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:25:05.568427    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:25:06.223873 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:06.223723    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:06.347768 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:06.347737    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"
Oct 30 22:25:06.415171 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:06.415125    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4d334339-1d4b-421b-856e-c2a0d9f4737a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d334339-1d4b-421b-856e-c2a0d9f4737a-istiod-ca-cert\") pod \"4d334339-1d4b-421b-856e-c2a0d9f4737a\" (UID: \"4d334339-1d4b-421b-856e-c2a0d9f4737a\") "
Oct 30 22:25:06.415416 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:06.415201    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d334339-1d4b-421b-856e-c2a0d9f4737a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d334339-1d4b-421b-856e-c2a0d9f4737a-workload-socket\") pod \"4d334339-1d4b-421b-856e-c2a0d9f4737a\" (UID: \"4d334339-1d4b-421b-856e-c2a0d9f4737a\") "
Oct 30 22:25:06.415416 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:06.415302    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d334339-1d4b-421b-856e-c2a0d9f4737a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d334339-1d4b-421b-856e-c2a0d9f4737a-credential-socket\") pod \"4d334339-1d4b-421b-856e-c2a0d9f4737a\" (UID: \"4d334339-1d4b-421b-856e-c2a0d9f4737a\") "
Oct 30 22:25:06.415416 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:06.415371    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4d334339-1d4b-421b-856e-c2a0d9f4737a-kube-api-access-f6fzx\" (UniqueName: \"kubernetes.io/projected/4d334339-1d4b-421b-856e-c2a0d9f4737a-kube-api-access-f6fzx\") pod \"4d334339-1d4b-421b-856e-c2a0d9f4737a\" (UID: \"4d334339-1d4b-421b-856e-c2a0d9f4737a\") "
Oct 30 22:25:06.415610 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:06.415426    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4d334339-1d4b-421b-856e-c2a0d9f4737a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d334339-1d4b-421b-856e-c2a0d9f4737a-istio-podinfo\") pod \"4d334339-1d4b-421b-856e-c2a0d9f4737a\" (UID: \"4d334339-1d4b-421b-856e-c2a0d9f4737a\") "
Oct 30 22:25:06.415610 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:06.415529    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4d334339-1d4b-421b-856e-c2a0d9f4737a-istio-token\" (UniqueName: \"kubernetes.io/projected/4d334339-1d4b-421b-856e-c2a0d9f4737a-istio-token\") pod \"4d334339-1d4b-421b-856e-c2a0d9f4737a\" (UID: \"4d334339-1d4b-421b-856e-c2a0d9f4737a\") "
Oct 30 22:25:06.415610 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:06.415554    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d334339-1d4b-421b-856e-c2a0d9f4737a-workload-socket" pod "4d334339-1d4b-421b-856e-c2a0d9f4737a" (UID: "4d334339-1d4b-421b-856e-c2a0d9f4737a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:25:06.415610 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:06.415570    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d334339-1d4b-421b-856e-c2a0d9f4737a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d334339-1d4b-421b-856e-c2a0d9f4737a-workload-certs\") pod \"4d334339-1d4b-421b-856e-c2a0d9f4737a\" (UID: \"4d334339-1d4b-421b-856e-c2a0d9f4737a\") "
Oct 30 22:25:06.415610 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:06.415589    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d334339-1d4b-421b-856e-c2a0d9f4737a-istiod-ca-cert" pod "4d334339-1d4b-421b-856e-c2a0d9f4737a" (UID: "4d334339-1d4b-421b-856e-c2a0d9f4737a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:25:06.415872 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:06.415650    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d334339-1d4b-421b-856e-c2a0d9f4737a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d334339-1d4b-421b-856e-c2a0d9f4737a-istio-data\") pod \"4d334339-1d4b-421b-856e-c2a0d9f4737a\" (UID: \"4d334339-1d4b-421b-856e-c2a0d9f4737a\") "
Oct 30 22:25:06.415872 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:06.415687    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d334339-1d4b-421b-856e-c2a0d9f4737a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d334339-1d4b-421b-856e-c2a0d9f4737a-istio-envoy\") pod \"4d334339-1d4b-421b-856e-c2a0d9f4737a\" (UID: \"4d334339-1d4b-421b-856e-c2a0d9f4737a\") "
Oct 30 22:25:06.415872 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:06.415790    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d334339-1d4b-421b-856e-c2a0d9f4737a-workload-certs" pod "4d334339-1d4b-421b-856e-c2a0d9f4737a" (UID: "4d334339-1d4b-421b-856e-c2a0d9f4737a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:25:06.416000 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:06.415907    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d334339-1d4b-421b-856e-c2a0d9f4737a-istio-data" pod "4d334339-1d4b-421b-856e-c2a0d9f4737a" (UID: "4d334339-1d4b-421b-856e-c2a0d9f4737a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:25:06.416316 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:06.416291    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d334339-1d4b-421b-856e-c2a0d9f4737a-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:06.416416 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:06.416300    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d334339-1d4b-421b-856e-c2a0d9f4737a-credential-socket" pod "4d334339-1d4b-421b-856e-c2a0d9f4737a" (UID: "4d334339-1d4b-421b-856e-c2a0d9f4737a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:25:06.416416 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:06.416318    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d334339-1d4b-421b-856e-c2a0d9f4737a-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:06.416416 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:06.416355    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d334339-1d4b-421b-856e-c2a0d9f4737a-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:06.416416 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:06.416376    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d334339-1d4b-421b-856e-c2a0d9f4737a-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:06.417939 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:06.417910    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d334339-1d4b-421b-856e-c2a0d9f4737a-istio-envoy" pod "4d334339-1d4b-421b-856e-c2a0d9f4737a" (UID: "4d334339-1d4b-421b-856e-c2a0d9f4737a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:25:06.418256 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:06.418213    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d334339-1d4b-421b-856e-c2a0d9f4737a-istio-token" pod "4d334339-1d4b-421b-856e-c2a0d9f4737a" (UID: "4d334339-1d4b-421b-856e-c2a0d9f4737a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:25:06.418396 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:06.418377    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d334339-1d4b-421b-856e-c2a0d9f4737a-kube-api-access-f6fzx" pod "4d334339-1d4b-421b-856e-c2a0d9f4737a" (UID: "4d334339-1d4b-421b-856e-c2a0d9f4737a"). InnerVolumeSpecName "kube-api-access-f6fzx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:25:06.418462 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:06.418431    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4d334339-1d4b-421b-856e-c2a0d9f4737a-istio-podinfo" pod "4d334339-1d4b-421b-856e-c2a0d9f4737a" (UID: "4d334339-1d4b-421b-856e-c2a0d9f4737a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:25:06.517695 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:06.517641    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d334339-1d4b-421b-856e-c2a0d9f4737a-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:06.517695 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:06.517690    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d334339-1d4b-421b-856e-c2a0d9f4737a-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:06.517695 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:06.517706    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d334339-1d4b-421b-856e-c2a0d9f4737a-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:06.517965 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:06.517721    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6fzx\" (UniqueName: \"kubernetes.io/projected/4d334339-1d4b-421b-856e-c2a0d9f4737a-kube-api-access-f6fzx\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:06.517965 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:06.517737    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d334339-1d4b-421b-856e-c2a0d9f4737a-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:06.804350 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:06.804309    4871 generic.go:356] "Generic (PLEG): container finished" podID="4d334339-1d4b-421b-856e-c2a0d9f4737a" containerID="849ada11e535fa4bffd8d26586363c6193392257e1aab2a58b45119ea3f615de" exitCode=0
Oct 30 22:25:06.804821 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:06.804361    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"
Oct 30 22:25:06.804821 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:06.804375    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g" event={"ID":"4d334339-1d4b-421b-856e-c2a0d9f4737a","Type":"ContainerDied","Data":"849ada11e535fa4bffd8d26586363c6193392257e1aab2a58b45119ea3f615de"}
Oct 30 22:25:06.804821 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:06.804430    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g" event={"ID":"4d334339-1d4b-421b-856e-c2a0d9f4737a","Type":"ContainerDied","Data":"7ed71ac2510dba28351383f4713b939f07ac53b29548f57bd9ae156994c0ad41"}
Oct 30 22:25:06.804821 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:06.804458    4871 scope.go:122] "RemoveContainer" containerID="849ada11e535fa4bffd8d26586363c6193392257e1aab2a58b45119ea3f615de"
Oct 30 22:25:06.818406 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:06.818380    4871 scope.go:122] "RemoveContainer" containerID="ad7e4c29bbccf337b323631161bab60909fe00c53884b029ad4567f3ea592ef6"
Oct 30 22:25:06.828967 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:06.828939    4871 scope.go:122] "RemoveContainer" containerID="849ada11e535fa4bffd8d26586363c6193392257e1aab2a58b45119ea3f615de"
Oct 30 22:25:06.829403 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:25:06.829361    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"849ada11e535fa4bffd8d26586363c6193392257e1aab2a58b45119ea3f615de\": container with ID starting with 849ada11e535fa4bffd8d26586363c6193392257e1aab2a58b45119ea3f615de not found: ID does not exist" containerID="849ada11e535fa4bffd8d26586363c6193392257e1aab2a58b45119ea3f615de"
Oct 30 22:25:06.829489 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:06.829417    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"849ada11e535fa4bffd8d26586363c6193392257e1aab2a58b45119ea3f615de"} err="failed to get container status \"849ada11e535fa4bffd8d26586363c6193392257e1aab2a58b45119ea3f615de\": rpc error: code = NotFound desc = could not find container \"849ada11e535fa4bffd8d26586363c6193392257e1aab2a58b45119ea3f615de\": container with ID starting with 849ada11e535fa4bffd8d26586363c6193392257e1aab2a58b45119ea3f615de not found: ID does not exist"
Oct 30 22:25:06.829489 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:06.829443    4871 scope.go:122] "RemoveContainer" containerID="ad7e4c29bbccf337b323631161bab60909fe00c53884b029ad4567f3ea592ef6"
Oct 30 22:25:06.829806 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:25:06.829778    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad7e4c29bbccf337b323631161bab60909fe00c53884b029ad4567f3ea592ef6\": container with ID starting with ad7e4c29bbccf337b323631161bab60909fe00c53884b029ad4567f3ea592ef6 not found: ID does not exist" containerID="ad7e4c29bbccf337b323631161bab60909fe00c53884b029ad4567f3ea592ef6"
Oct 30 22:25:06.829902 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:06.829821    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad7e4c29bbccf337b323631161bab60909fe00c53884b029ad4567f3ea592ef6"} err="failed to get container status \"ad7e4c29bbccf337b323631161bab60909fe00c53884b029ad4567f3ea592ef6\": rpc error: code = NotFound desc = could not find container \"ad7e4c29bbccf337b323631161bab60909fe00c53884b029ad4567f3ea592ef6\": container with ID starting with ad7e4c29bbccf337b323631161bab60909fe00c53884b029ad4567f3ea592ef6 not found: ID does not exist"
Oct 30 22:25:06.978564 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:06.978526    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"]
Oct 30 22:25:07.036148 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:07.034481    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-a-ca6d0b-openshift-default-7677b768f7-cgh8g"]
Oct 30 22:25:07.962359 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:07.962323    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4d334339-1d4b-421b-856e-c2a0d9f4737a" path="/var/lib/kubelet/pods/4d334339-1d4b-421b-856e-c2a0d9f4737a/volumes"
Oct 30 22:25:07.985040 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:07.985008    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:08.149078 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:08.149046    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:08.347467 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:08.347355    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"]
Oct 30 22:25:08.347656 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:08.347591    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5" podUID="d6407e4e-8850-4c67-8ad5-93c7fde3ea55" containerName="istio-proxy" containerID="cri-o://c5b02f4153df0425fe27f39a39c9bccc08b77151c916e9c551aa887059fd0092" gracePeriod=30
Oct 30 22:25:08.357301 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:08.357273    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:10.844852 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:10.844814    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:13.019588 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.019552    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-fc3918-585f79b757-nzmct"]
Oct 30 22:25:13.395106 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.395069    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:13.497741 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.497707    4871 patch_prober.go:29] interesting pod/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused" start-of-body=
Oct 30 22:25:13.497903 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.497766    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5" podUID="d6407e4e-8850-4c67-8ad5-93c7fde3ea55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused"
Oct 30 22:25:13.552917 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.552881    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:13.717827 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.717786    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"
Oct 30 22:25:13.821260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.816308    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-credential-socket\") pod \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\" (UID: \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\") "
Oct 30 22:25:13.821260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.816431    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-istio-podinfo\") pod \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\" (UID: \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\") "
Oct 30 22:25:13.821260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.816484    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-workload-socket\") pod \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\" (UID: \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\") "
Oct 30 22:25:13.821260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.816530    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-workload-certs\") pod \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\" (UID: \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\") "
Oct 30 22:25:13.821260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.816577    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-istio-envoy\") pod \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\" (UID: \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\") "
Oct 30 22:25:13.821260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.816606    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-istiod-ca-cert\") pod \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\" (UID: \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\") "
Oct 30 22:25:13.821260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.816674    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-kube-api-access-44wqh\" (UniqueName: \"kubernetes.io/projected/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-kube-api-access-44wqh\") pod \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\" (UID: \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\") "
Oct 30 22:25:13.821260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.816739    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-istio-token\" (UniqueName: \"kubernetes.io/projected/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-istio-token\") pod \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\" (UID: \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\") "
Oct 30 22:25:13.821260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.816781    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-istio-data\") pod \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\" (UID: \"d6407e4e-8850-4c67-8ad5-93c7fde3ea55\") "
Oct 30 22:25:13.821260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.817696    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-istio-data" pod "d6407e4e-8850-4c67-8ad5-93c7fde3ea55" (UID: "d6407e4e-8850-4c67-8ad5-93c7fde3ea55"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:25:13.821260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.817922    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-credential-socket" pod "d6407e4e-8850-4c67-8ad5-93c7fde3ea55" (UID: "d6407e4e-8850-4c67-8ad5-93c7fde3ea55"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:25:13.821260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.820777    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-workload-socket" pod "d6407e4e-8850-4c67-8ad5-93c7fde3ea55" (UID: "d6407e4e-8850-4c67-8ad5-93c7fde3ea55"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:25:13.821260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.821095    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-istiod-ca-cert" pod "d6407e4e-8850-4c67-8ad5-93c7fde3ea55" (UID: "d6407e4e-8850-4c67-8ad5-93c7fde3ea55"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:25:13.821260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.821092    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-workload-certs" pod "d6407e4e-8850-4c67-8ad5-93c7fde3ea55" (UID: "d6407e4e-8850-4c67-8ad5-93c7fde3ea55"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:25:13.821260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.821172    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-istio-podinfo" pod "d6407e4e-8850-4c67-8ad5-93c7fde3ea55" (UID: "d6407e4e-8850-4c67-8ad5-93c7fde3ea55"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:25:13.854769 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.854725    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-kube-api-access-44wqh" pod "d6407e4e-8850-4c67-8ad5-93c7fde3ea55" (UID: "d6407e4e-8850-4c67-8ad5-93c7fde3ea55"). InnerVolumeSpecName "kube-api-access-44wqh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:25:13.858527 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.858487    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-istio-token" pod "d6407e4e-8850-4c67-8ad5-93c7fde3ea55" (UID: "d6407e4e-8850-4c67-8ad5-93c7fde3ea55"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:25:13.860623 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.860588    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-istio-envoy" pod "d6407e4e-8850-4c67-8ad5-93c7fde3ea55" (UID: "d6407e4e-8850-4c67-8ad5-93c7fde3ea55"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:25:13.871426 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.871389    4871 generic.go:356] "Generic (PLEG): container finished" podID="d6407e4e-8850-4c67-8ad5-93c7fde3ea55" containerID="c5b02f4153df0425fe27f39a39c9bccc08b77151c916e9c551aa887059fd0092" exitCode=0
Oct 30 22:25:13.871588 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.871461    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5" event={"ID":"d6407e4e-8850-4c67-8ad5-93c7fde3ea55","Type":"ContainerDied","Data":"c5b02f4153df0425fe27f39a39c9bccc08b77151c916e9c551aa887059fd0092"}
Oct 30 22:25:13.871588 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.871528    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5" event={"ID":"d6407e4e-8850-4c67-8ad5-93c7fde3ea55","Type":"ContainerDied","Data":"b3e1d0c483d523cf1e8e923539a99fea89bd8390ac8f6ad596cca21976f34829"}
Oct 30 22:25:13.871588 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.871557    4871 scope.go:122] "RemoveContainer" containerID="c5b02f4153df0425fe27f39a39c9bccc08b77151c916e9c551aa887059fd0092"
Oct 30 22:25:13.872524 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.872499    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"
Oct 30 22:25:13.896606 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.893552    4871 scope.go:122] "RemoveContainer" containerID="ff5ec537f48ff74c1c24133feccb802e9c75d66e1b69adb617f0d0a9543743c3"
Oct 30 22:25:13.919227 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.919183    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:13.919227 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.919226    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:13.919507 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.919283    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:13.919507 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.919298    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:13.919507 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.919312    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:13.919507 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.919325    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:13.919507 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.919339    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44wqh\" (UniqueName: \"kubernetes.io/projected/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-kube-api-access-44wqh\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:13.919507 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.919354    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:13.919507 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.919367    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6407e4e-8850-4c67-8ad5-93c7fde3ea55-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:13.923391 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.923358    4871 scope.go:122] "RemoveContainer" containerID="c5b02f4153df0425fe27f39a39c9bccc08b77151c916e9c551aa887059fd0092"
Oct 30 22:25:13.923811 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:25:13.923784    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5b02f4153df0425fe27f39a39c9bccc08b77151c916e9c551aa887059fd0092\": container with ID starting with c5b02f4153df0425fe27f39a39c9bccc08b77151c916e9c551aa887059fd0092 not found: ID does not exist" containerID="c5b02f4153df0425fe27f39a39c9bccc08b77151c916e9c551aa887059fd0092"
Oct 30 22:25:13.923909 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.923820    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5b02f4153df0425fe27f39a39c9bccc08b77151c916e9c551aa887059fd0092"} err="failed to get container status \"c5b02f4153df0425fe27f39a39c9bccc08b77151c916e9c551aa887059fd0092\": rpc error: code = NotFound desc = could not find container \"c5b02f4153df0425fe27f39a39c9bccc08b77151c916e9c551aa887059fd0092\": container with ID starting with c5b02f4153df0425fe27f39a39c9bccc08b77151c916e9c551aa887059fd0092 not found: ID does not exist"
Oct 30 22:25:13.923909 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.923843    4871 scope.go:122] "RemoveContainer" containerID="ff5ec537f48ff74c1c24133feccb802e9c75d66e1b69adb617f0d0a9543743c3"
Oct 30 22:25:13.924218 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:25:13.924196    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff5ec537f48ff74c1c24133feccb802e9c75d66e1b69adb617f0d0a9543743c3\": container with ID starting with ff5ec537f48ff74c1c24133feccb802e9c75d66e1b69adb617f0d0a9543743c3 not found: ID does not exist" containerID="ff5ec537f48ff74c1c24133feccb802e9c75d66e1b69adb617f0d0a9543743c3"
Oct 30 22:25:13.924749 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:13.924222    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff5ec537f48ff74c1c24133feccb802e9c75d66e1b69adb617f0d0a9543743c3"} err="failed to get container status \"ff5ec537f48ff74c1c24133feccb802e9c75d66e1b69adb617f0d0a9543743c3\": rpc error: code = NotFound desc = could not find container \"ff5ec537f48ff74c1c24133feccb802e9c75d66e1b69adb617f0d0a9543743c3\": container with ID starting with ff5ec537f48ff74c1c24133feccb802e9c75d66e1b69adb617f0d0a9543743c3 not found: ID does not exist"
Oct 30 22:25:14.049457 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:14.049377    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"]
Oct 30 22:25:14.054085 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:14.054052    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-a-f4a424-openshift-default-79f48d97b5-sgwh5"]
Oct 30 22:25:15.536903 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:15.536861    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:15.960534 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:15.960497    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d6407e4e-8850-4c67-8ad5-93c7fde3ea55" path="/var/lib/kubelet/pods/d6407e4e-8850-4c67-8ad5-93c7fde3ea55/volumes"
Oct 30 22:25:18.020351 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:18.020286    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-praxis-b-fc3918-585f79b757-nzmct" podUID="914386df-d6fd-4b9d-92b9-45a987e3b514" containerName="payload-processing" containerID="cri-o://4eca8ba18babdd0d2848d35422e760cdb90315d7dbc04372d2e35a15b740b291" gracePeriod=25
Oct 30 22:25:18.291017 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:18.290989    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-fc3918-585f79b757-nzmct"
Oct 30 22:25:18.381558 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:18.381521    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/914386df-d6fd-4b9d-92b9-45a987e3b514-kube-api-access-8rc5g\" (UniqueName: \"kubernetes.io/projected/914386df-d6fd-4b9d-92b9-45a987e3b514-kube-api-access-8rc5g\") pod \"914386df-d6fd-4b9d-92b9-45a987e3b514\" (UID: \"914386df-d6fd-4b9d-92b9-45a987e3b514\") "
Oct 30 22:25:18.381722 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:18.381600    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/914386df-d6fd-4b9d-92b9-45a987e3b514-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/914386df-d6fd-4b9d-92b9-45a987e3b514-plugins-config-volume\") pod \"914386df-d6fd-4b9d-92b9-45a987e3b514\" (UID: \"914386df-d6fd-4b9d-92b9-45a987e3b514\") "
Oct 30 22:25:18.382015 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:18.381986    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/914386df-d6fd-4b9d-92b9-45a987e3b514-plugins-config-volume" pod "914386df-d6fd-4b9d-92b9-45a987e3b514" (UID: "914386df-d6fd-4b9d-92b9-45a987e3b514"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:25:18.383788 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:18.383761    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/914386df-d6fd-4b9d-92b9-45a987e3b514-kube-api-access-8rc5g" pod "914386df-d6fd-4b9d-92b9-45a987e3b514" (UID: "914386df-d6fd-4b9d-92b9-45a987e3b514"). InnerVolumeSpecName "kube-api-access-8rc5g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:25:18.483802 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:18.483743    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rc5g\" (UniqueName: \"kubernetes.io/projected/914386df-d6fd-4b9d-92b9-45a987e3b514-kube-api-access-8rc5g\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:18.483802 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:18.483798    4871 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/914386df-d6fd-4b9d-92b9-45a987e3b514-plugins-config-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:18.911997 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:18.911953    4871 generic.go:356] "Generic (PLEG): container finished" podID="914386df-d6fd-4b9d-92b9-45a987e3b514" containerID="4eca8ba18babdd0d2848d35422e760cdb90315d7dbc04372d2e35a15b740b291" exitCode=0
Oct 30 22:25:18.912190 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:18.912033    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-fc3918-585f79b757-nzmct" event={"ID":"914386df-d6fd-4b9d-92b9-45a987e3b514","Type":"ContainerDied","Data":"4eca8ba18babdd0d2848d35422e760cdb90315d7dbc04372d2e35a15b740b291"}
Oct 30 22:25:18.912190 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:18.912046    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-fc3918-585f79b757-nzmct"
Oct 30 22:25:18.912190 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:18.912174    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-fc3918-585f79b757-nzmct" event={"ID":"914386df-d6fd-4b9d-92b9-45a987e3b514","Type":"ContainerDied","Data":"8851a47a9a5c8f656308913965c155e360c08666577c01873c6751ba67830e79"}
Oct 30 22:25:18.912374 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:18.912212    4871 scope.go:122] "RemoveContainer" containerID="4eca8ba18babdd0d2848d35422e760cdb90315d7dbc04372d2e35a15b740b291"
Oct 30 22:25:18.925529 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:18.925482    4871 scope.go:122] "RemoveContainer" containerID="4eca8ba18babdd0d2848d35422e760cdb90315d7dbc04372d2e35a15b740b291"
Oct 30 22:25:18.925887 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:25:18.925864    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4eca8ba18babdd0d2848d35422e760cdb90315d7dbc04372d2e35a15b740b291\": container with ID starting with 4eca8ba18babdd0d2848d35422e760cdb90315d7dbc04372d2e35a15b740b291 not found: ID does not exist" containerID="4eca8ba18babdd0d2848d35422e760cdb90315d7dbc04372d2e35a15b740b291"
Oct 30 22:25:18.925942 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:18.925900    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4eca8ba18babdd0d2848d35422e760cdb90315d7dbc04372d2e35a15b740b291"} err="failed to get container status \"4eca8ba18babdd0d2848d35422e760cdb90315d7dbc04372d2e35a15b740b291\": rpc error: code = NotFound desc = could not find container \"4eca8ba18babdd0d2848d35422e760cdb90315d7dbc04372d2e35a15b740b291\": container with ID starting with 4eca8ba18babdd0d2848d35422e760cdb90315d7dbc04372d2e35a15b740b291 not found: ID does not exist"
Oct 30 22:25:19.086310 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:19.086275    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-fc3918-585f79b757-nzmct"]
Oct 30 22:25:19.139591 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:19.139551    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-fc3918-585f79b757-nzmct"]
Oct 30 22:25:19.960917 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:19.960872    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="914386df-d6fd-4b9d-92b9-45a987e3b514" path="/var/lib/kubelet/pods/914386df-d6fd-4b9d-92b9-45a987e3b514/volumes"
Oct 30 22:25:20.161436 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:20.161401    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:25.019411 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:25.019372    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:28.568576 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:28.568539    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:29.614588 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:29.614555    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:31.952959 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:31.952921    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:32.132165 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:32.132123    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:34.317660 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:34.317625    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:35.568886 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:25:35.568845    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:25:38.913051 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:38.913009    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:43.622700 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:43.622666    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:45.110981 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:45.110939    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:45.363551 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:45.363510    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:46.554473 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:46.554437    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:48.310630 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:48.310591    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:53.903780 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:53.903745    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:54.298305 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:54.298181    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:55.387332 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:55.387297    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:55.615618 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:55.615581    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:55.771827 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:55.771733    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:56.301617 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:25:56.301584    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:00.410151 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:00.410112    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:02.624438 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:02.624405    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:02.821648 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:02.821611    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:03.014703 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:03.014621    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:05.100098 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:05.100069    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:05.568764 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:26:05.568721    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:26:08.960015 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:08.959980    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2"]
Oct 30 22:26:08.960440 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:08.960210    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2" podUID="5d03255e-3722-45d9-b21b-b1c751355173" containerName="maas-api" containerID="cri-o://990490fa7c4094eb832fbf2b1722244814add3160ec289dd8a6bcdaab41509e7" gracePeriod=30
Oct 30 22:26:09.094632 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:09.094586    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:09.295753 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:09.295721    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2"
Oct 30 22:26:09.321690 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:09.321643    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5d03255e-3722-45d9-b21b-b1c751355173-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d03255e-3722-45d9-b21b-b1c751355173-maas-api-tls\") pod \"5d03255e-3722-45d9-b21b-b1c751355173\" (UID: \"5d03255e-3722-45d9-b21b-b1c751355173\") "
Oct 30 22:26:09.321879 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:09.321709    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5d03255e-3722-45d9-b21b-b1c751355173-kube-api-access-nk4x6\" (UniqueName: \"kubernetes.io/projected/5d03255e-3722-45d9-b21b-b1c751355173-kube-api-access-nk4x6\") pod \"5d03255e-3722-45d9-b21b-b1c751355173\" (UID: \"5d03255e-3722-45d9-b21b-b1c751355173\") "
Oct 30 22:26:09.321879 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:09.321830    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5d03255e-3722-45d9-b21b-b1c751355173-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d03255e-3722-45d9-b21b-b1c751355173-openshift-service-ca\") pod \"5d03255e-3722-45d9-b21b-b1c751355173\" (UID: \"5d03255e-3722-45d9-b21b-b1c751355173\") "
Oct 30 22:26:09.321879 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:09.321869    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5d03255e-3722-45d9-b21b-b1c751355173-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d03255e-3722-45d9-b21b-b1c751355173-metrics-certs\") pod \"5d03255e-3722-45d9-b21b-b1c751355173\" (UID: \"5d03255e-3722-45d9-b21b-b1c751355173\") "
Oct 30 22:26:09.322214 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:09.322185    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d03255e-3722-45d9-b21b-b1c751355173-openshift-service-ca" pod "5d03255e-3722-45d9-b21b-b1c751355173" (UID: "5d03255e-3722-45d9-b21b-b1c751355173"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:26:09.324161 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:09.324123    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d03255e-3722-45d9-b21b-b1c751355173-metrics-certs" pod "5d03255e-3722-45d9-b21b-b1c751355173" (UID: "5d03255e-3722-45d9-b21b-b1c751355173"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:26:09.324349 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:09.324187    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d03255e-3722-45d9-b21b-b1c751355173-kube-api-access-nk4x6" pod "5d03255e-3722-45d9-b21b-b1c751355173" (UID: "5d03255e-3722-45d9-b21b-b1c751355173"). InnerVolumeSpecName "kube-api-access-nk4x6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:26:09.324349 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:09.324269    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d03255e-3722-45d9-b21b-b1c751355173-maas-api-tls" pod "5d03255e-3722-45d9-b21b-b1c751355173" (UID: "5d03255e-3722-45d9-b21b-b1c751355173"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:26:09.383264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:09.383195    4871 generic.go:356] "Generic (PLEG): container finished" podID="5d03255e-3722-45d9-b21b-b1c751355173" containerID="990490fa7c4094eb832fbf2b1722244814add3160ec289dd8a6bcdaab41509e7" exitCode=0
Oct 30 22:26:09.383472 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:09.383288    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2" event={"ID":"5d03255e-3722-45d9-b21b-b1c751355173","Type":"ContainerDied","Data":"990490fa7c4094eb832fbf2b1722244814add3160ec289dd8a6bcdaab41509e7"}
Oct 30 22:26:09.383472 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:09.383343    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2" event={"ID":"5d03255e-3722-45d9-b21b-b1c751355173","Type":"ContainerDied","Data":"15f641577b849fbc6eb9e5f43aed8261903ae158c0c4a93ab8437d339c9c9cb8"}
Oct 30 22:26:09.383472 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:09.383374    4871 scope.go:122] "RemoveContainer" containerID="990490fa7c4094eb832fbf2b1722244814add3160ec289dd8a6bcdaab41509e7"
Oct 30 22:26:09.383621 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:09.383542    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2"
Oct 30 22:26:09.398769 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:09.398744    4871 scope.go:122] "RemoveContainer" containerID="990490fa7c4094eb832fbf2b1722244814add3160ec289dd8a6bcdaab41509e7"
Oct 30 22:26:09.399098 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:26:09.399071    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"990490fa7c4094eb832fbf2b1722244814add3160ec289dd8a6bcdaab41509e7\": container with ID starting with 990490fa7c4094eb832fbf2b1722244814add3160ec289dd8a6bcdaab41509e7 not found: ID does not exist" containerID="990490fa7c4094eb832fbf2b1722244814add3160ec289dd8a6bcdaab41509e7"
Oct 30 22:26:09.399150 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:09.399107    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"990490fa7c4094eb832fbf2b1722244814add3160ec289dd8a6bcdaab41509e7"} err="failed to get container status \"990490fa7c4094eb832fbf2b1722244814add3160ec289dd8a6bcdaab41509e7\": rpc error: code = NotFound desc = could not find container \"990490fa7c4094eb832fbf2b1722244814add3160ec289dd8a6bcdaab41509e7\": container with ID starting with 990490fa7c4094eb832fbf2b1722244814add3160ec289dd8a6bcdaab41509e7 not found: ID does not exist"
Oct 30 22:26:09.423413 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:09.423372    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d03255e-3722-45d9-b21b-b1c751355173-maas-api-tls\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:09.423413 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:09.423413    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nk4x6\" (UniqueName: \"kubernetes.io/projected/5d03255e-3722-45d9-b21b-b1c751355173-kube-api-access-nk4x6\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:09.423413 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:09.423428    4871 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d03255e-3722-45d9-b21b-b1c751355173-openshift-service-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:09.423744 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:09.423441    4871 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d03255e-3722-45d9-b21b-b1c751355173-metrics-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:09.554076 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:09.553967    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2"]
Oct 30 22:26:09.557337 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:09.557309    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-93131d-5588d54b48-f5kx2"]
Oct 30 22:26:09.963303 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:09.963268    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5d03255e-3722-45d9-b21b-b1c751355173" path="/var/lib/kubelet/pods/5d03255e-3722-45d9-b21b-b1c751355173/volumes"
Oct 30 22:26:10.597600 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:10.597566    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:10.896730 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:10.896693    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:11.054793 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:11.054758    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:11.621605 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:11.621571    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:15.950076 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:15.950043    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:18.042462 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:18.042365    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:18.268597 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:18.268558    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:18.417903 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:18.417871    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"]
Oct 30 22:26:18.418106 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:18.418073    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9" podUID="689c9d7f-c8ab-49e5-b647-b4a62d97c1e7" containerName="istio-proxy" containerID="cri-o://15458c6d8fcd1a410d6d8a7124ea8545ba3d15d34e29f452ca03a24b027c395f" gracePeriod=30
Oct 30 22:26:18.465667 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:18.465619    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:20.029903 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:20.029855    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:20.184901 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:20.184845    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:20.506505 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:20.506464    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:23.588046 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:23.587998    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"]
Oct 30 22:26:23.588634 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:23.588295    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8" podUID="1d075c21-db31-400b-b173-02391280ad9f" containerName="istio-proxy" containerID="cri-o://3df5dfb85d51185b3f122c9e766848737c76dc07154a01a0fb0f093f72a62067" gracePeriod=30
Oct 30 22:26:23.745956 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:23.745925    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"
Oct 30 22:26:23.822340 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:23.822305    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-istio-podinfo\") pod \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\" (UID: \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\") "
Oct 30 22:26:23.822533 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:23.822385    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-workload-socket\") pod \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\" (UID: \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\") "
Oct 30 22:26:23.822533 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:23.822448    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-istio-token\" (UniqueName: \"kubernetes.io/projected/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-istio-token\") pod \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\" (UID: \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\") "
Oct 30 22:26:23.822533 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:23.822498    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-workload-certs\") pod \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\" (UID: \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\") "
Oct 30 22:26:23.822667 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:23.822597    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-kube-api-access-t4brw\" (UniqueName: \"kubernetes.io/projected/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-kube-api-access-t4brw\") pod \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\" (UID: \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\") "
Oct 30 22:26:23.822667 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:23.822643    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-istio-data\") pod \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\" (UID: \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\") "
Oct 30 22:26:23.822773 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:23.822708    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-istio-envoy\") pod \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\" (UID: \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\") "
Oct 30 22:26:23.822773 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:23.822735    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-workload-socket" pod "689c9d7f-c8ab-49e5-b647-b4a62d97c1e7" (UID: "689c9d7f-c8ab-49e5-b647-b4a62d97c1e7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:23.822773 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:23.822746    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-istiod-ca-cert\") pod \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\" (UID: \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\") "
Oct 30 22:26:23.822950 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:23.822840    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-credential-socket\") pod \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\" (UID: \"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7\") "
Oct 30 22:26:23.823035 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:23.822999    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-workload-certs" pod "689c9d7f-c8ab-49e5-b647-b4a62d97c1e7" (UID: "689c9d7f-c8ab-49e5-b647-b4a62d97c1e7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:23.823108 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:23.823071    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-istiod-ca-cert" pod "689c9d7f-c8ab-49e5-b647-b4a62d97c1e7" (UID: "689c9d7f-c8ab-49e5-b647-b4a62d97c1e7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:26:23.823320 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:23.823295    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-credential-socket" pod "689c9d7f-c8ab-49e5-b647-b4a62d97c1e7" (UID: "689c9d7f-c8ab-49e5-b647-b4a62d97c1e7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:23.823554 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:23.823529    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:23.823638 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:23.823561    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:23.823638 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:23.823579    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:23.823638 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:23.823594    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:23.823882 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:23.823857    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-istio-data" pod "689c9d7f-c8ab-49e5-b647-b4a62d97c1e7" (UID: "689c9d7f-c8ab-49e5-b647-b4a62d97c1e7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:23.824954 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:23.824921    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-istio-podinfo" pod "689c9d7f-c8ab-49e5-b647-b4a62d97c1e7" (UID: "689c9d7f-c8ab-49e5-b647-b4a62d97c1e7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:26:23.825379 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:23.825360    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-istio-token" pod "689c9d7f-c8ab-49e5-b647-b4a62d97c1e7" (UID: "689c9d7f-c8ab-49e5-b647-b4a62d97c1e7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:26:23.825636 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:23.825620    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-istio-envoy" pod "689c9d7f-c8ab-49e5-b647-b4a62d97c1e7" (UID: "689c9d7f-c8ab-49e5-b647-b4a62d97c1e7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:23.825722 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:23.825691    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-kube-api-access-t4brw" pod "689c9d7f-c8ab-49e5-b647-b4a62d97c1e7" (UID: "689c9d7f-c8ab-49e5-b647-b4a62d97c1e7"). InnerVolumeSpecName "kube-api-access-t4brw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:26:23.924635 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:23.924594    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4brw\" (UniqueName: \"kubernetes.io/projected/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-kube-api-access-t4brw\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:23.924635 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:23.924632    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:23.924635 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:23.924641    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:23.924899 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:23.924650    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:23.924899 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:23.924659    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:24.407097 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:24.407055    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn"]
Oct 30 22:26:24.407383 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:24.407355    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn" podUID="aec98483-f8d7-4eb3-811e-5f6b86e56230" containerName="maas-api" containerID="cri-o://1e5659f43eee7a0d449a6c665a4f99636ab1cef7504bc6dbe1e592640e45314f" gracePeriod=30
Oct 30 22:26:24.504885 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:24.504844    4871 generic.go:356] "Generic (PLEG): container finished" podID="689c9d7f-c8ab-49e5-b647-b4a62d97c1e7" containerID="15458c6d8fcd1a410d6d8a7124ea8545ba3d15d34e29f452ca03a24b027c395f" exitCode=0
Oct 30 22:26:24.505071 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:24.504930    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"
Oct 30 22:26:24.505071 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:24.504924    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9" event={"ID":"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7","Type":"ContainerDied","Data":"15458c6d8fcd1a410d6d8a7124ea8545ba3d15d34e29f452ca03a24b027c395f"}
Oct 30 22:26:24.505210 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:24.505070    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9" event={"ID":"689c9d7f-c8ab-49e5-b647-b4a62d97c1e7","Type":"ContainerDied","Data":"5143f3600708be81db52c4ef054bea4a21100bcf3f0a125aba69748cb686fe85"}
Oct 30 22:26:24.505210 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:24.505101    4871 scope.go:122] "RemoveContainer" containerID="15458c6d8fcd1a410d6d8a7124ea8545ba3d15d34e29f452ca03a24b027c395f"
Oct 30 22:26:24.523861 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:24.523833    4871 scope.go:122] "RemoveContainer" containerID="236a45bbecdb97b8c9ab767753bc6e8ed506280be126a9fb0cf863f326e843a5"
Oct 30 22:26:24.524387 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:24.524363    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:24.535308 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:24.535277    4871 scope.go:122] "RemoveContainer" containerID="15458c6d8fcd1a410d6d8a7124ea8545ba3d15d34e29f452ca03a24b027c395f"
Oct 30 22:26:24.535680 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:26:24.535657    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15458c6d8fcd1a410d6d8a7124ea8545ba3d15d34e29f452ca03a24b027c395f\": container with ID starting with 15458c6d8fcd1a410d6d8a7124ea8545ba3d15d34e29f452ca03a24b027c395f not found: ID does not exist" containerID="15458c6d8fcd1a410d6d8a7124ea8545ba3d15d34e29f452ca03a24b027c395f"
Oct 30 22:26:24.535764 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:24.535688    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15458c6d8fcd1a410d6d8a7124ea8545ba3d15d34e29f452ca03a24b027c395f"} err="failed to get container status \"15458c6d8fcd1a410d6d8a7124ea8545ba3d15d34e29f452ca03a24b027c395f\": rpc error: code = NotFound desc = could not find container \"15458c6d8fcd1a410d6d8a7124ea8545ba3d15d34e29f452ca03a24b027c395f\": container with ID starting with 15458c6d8fcd1a410d6d8a7124ea8545ba3d15d34e29f452ca03a24b027c395f not found: ID does not exist"
Oct 30 22:26:24.535764 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:24.535709    4871 scope.go:122] "RemoveContainer" containerID="236a45bbecdb97b8c9ab767753bc6e8ed506280be126a9fb0cf863f326e843a5"
Oct 30 22:26:24.536018 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:26:24.535995    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"236a45bbecdb97b8c9ab767753bc6e8ed506280be126a9fb0cf863f326e843a5\": container with ID starting with 236a45bbecdb97b8c9ab767753bc6e8ed506280be126a9fb0cf863f326e843a5 not found: ID does not exist" containerID="236a45bbecdb97b8c9ab767753bc6e8ed506280be126a9fb0cf863f326e843a5"
Oct 30 22:26:24.536092 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:24.536030    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"236a45bbecdb97b8c9ab767753bc6e8ed506280be126a9fb0cf863f326e843a5"} err="failed to get container status \"236a45bbecdb97b8c9ab767753bc6e8ed506280be126a9fb0cf863f326e843a5\": rpc error: code = NotFound desc = could not find container \"236a45bbecdb97b8c9ab767753bc6e8ed506280be126a9fb0cf863f326e843a5\": container with ID starting with 236a45bbecdb97b8c9ab767753bc6e8ed506280be126a9fb0cf863f326e843a5 not found: ID does not exist"
Oct 30 22:26:24.669758 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:24.669659    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"]
Oct 30 22:26:24.675504 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:24.675458    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-b-93131d-openshift-default-6b4b88f469-977r9"]
Oct 30 22:26:24.732488 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:24.732457    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn"
Oct 30 22:26:24.836917 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:24.836823    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aec98483-f8d7-4eb3-811e-5f6b86e56230-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aec98483-f8d7-4eb3-811e-5f6b86e56230-maas-api-tls\") pod \"aec98483-f8d7-4eb3-811e-5f6b86e56230\" (UID: \"aec98483-f8d7-4eb3-811e-5f6b86e56230\") "
Oct 30 22:26:24.837107 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:24.837034    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aec98483-f8d7-4eb3-811e-5f6b86e56230-kube-api-access-kfjhs\" (UniqueName: \"kubernetes.io/projected/aec98483-f8d7-4eb3-811e-5f6b86e56230-kube-api-access-kfjhs\") pod \"aec98483-f8d7-4eb3-811e-5f6b86e56230\" (UID: \"aec98483-f8d7-4eb3-811e-5f6b86e56230\") "
Oct 30 22:26:24.837178 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:24.837131    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aec98483-f8d7-4eb3-811e-5f6b86e56230-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aec98483-f8d7-4eb3-811e-5f6b86e56230-openshift-service-ca\") pod \"aec98483-f8d7-4eb3-811e-5f6b86e56230\" (UID: \"aec98483-f8d7-4eb3-811e-5f6b86e56230\") "
Oct 30 22:26:24.837217 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:24.837176    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aec98483-f8d7-4eb3-811e-5f6b86e56230-metrics-certs\" (UniqueName: \"kubernetes.io/secret/aec98483-f8d7-4eb3-811e-5f6b86e56230-metrics-certs\") pod \"aec98483-f8d7-4eb3-811e-5f6b86e56230\" (UID: \"aec98483-f8d7-4eb3-811e-5f6b86e56230\") "
Oct 30 22:26:24.838291 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:24.838261    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aec98483-f8d7-4eb3-811e-5f6b86e56230-openshift-service-ca" pod "aec98483-f8d7-4eb3-811e-5f6b86e56230" (UID: "aec98483-f8d7-4eb3-811e-5f6b86e56230"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:26:24.840343 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:24.840298    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aec98483-f8d7-4eb3-811e-5f6b86e56230-metrics-certs" pod "aec98483-f8d7-4eb3-811e-5f6b86e56230" (UID: "aec98483-f8d7-4eb3-811e-5f6b86e56230"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:26:24.841284 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:24.841247    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aec98483-f8d7-4eb3-811e-5f6b86e56230-maas-api-tls" pod "aec98483-f8d7-4eb3-811e-5f6b86e56230" (UID: "aec98483-f8d7-4eb3-811e-5f6b86e56230"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:26:24.843623 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:24.843597    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aec98483-f8d7-4eb3-811e-5f6b86e56230-kube-api-access-kfjhs" pod "aec98483-f8d7-4eb3-811e-5f6b86e56230" (UID: "aec98483-f8d7-4eb3-811e-5f6b86e56230"). InnerVolumeSpecName "kube-api-access-kfjhs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:26:24.938714 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:24.938626    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kfjhs\" (UniqueName: \"kubernetes.io/projected/aec98483-f8d7-4eb3-811e-5f6b86e56230-kube-api-access-kfjhs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:24.938714 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:24.938659    4871 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aec98483-f8d7-4eb3-811e-5f6b86e56230-openshift-service-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:24.938714 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:24.938670    4871 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aec98483-f8d7-4eb3-811e-5f6b86e56230-metrics-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:24.938714 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:24.938679    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aec98483-f8d7-4eb3-811e-5f6b86e56230-maas-api-tls\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:25.024696 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:25.024623    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:25.181038 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:25.180998    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:25.514630 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:25.514596    4871 generic.go:356] "Generic (PLEG): container finished" podID="aec98483-f8d7-4eb3-811e-5f6b86e56230" containerID="1e5659f43eee7a0d449a6c665a4f99636ab1cef7504bc6dbe1e592640e45314f" exitCode=0
Oct 30 22:26:25.514868 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:25.514663    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn"
Oct 30 22:26:25.514868 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:25.514671    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn" event={"ID":"aec98483-f8d7-4eb3-811e-5f6b86e56230","Type":"ContainerDied","Data":"1e5659f43eee7a0d449a6c665a4f99636ab1cef7504bc6dbe1e592640e45314f"}
Oct 30 22:26:25.514868 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:25.514813    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn" event={"ID":"aec98483-f8d7-4eb3-811e-5f6b86e56230","Type":"ContainerDied","Data":"777e50887d2b51f53f1c6e45152ef7caa12ea93353a1c8ba61549e5327270ee3"}
Oct 30 22:26:25.514868 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:25.514852    4871 scope.go:122] "RemoveContainer" containerID="1e5659f43eee7a0d449a6c665a4f99636ab1cef7504bc6dbe1e592640e45314f"
Oct 30 22:26:25.527398 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:25.527375    4871 scope.go:122] "RemoveContainer" containerID="1e5659f43eee7a0d449a6c665a4f99636ab1cef7504bc6dbe1e592640e45314f"
Oct 30 22:26:25.527709 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:26:25.527690    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e5659f43eee7a0d449a6c665a4f99636ab1cef7504bc6dbe1e592640e45314f\": container with ID starting with 1e5659f43eee7a0d449a6c665a4f99636ab1cef7504bc6dbe1e592640e45314f not found: ID does not exist" containerID="1e5659f43eee7a0d449a6c665a4f99636ab1cef7504bc6dbe1e592640e45314f"
Oct 30 22:26:25.527780 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:25.527720    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e5659f43eee7a0d449a6c665a4f99636ab1cef7504bc6dbe1e592640e45314f"} err="failed to get container status \"1e5659f43eee7a0d449a6c665a4f99636ab1cef7504bc6dbe1e592640e45314f\": rpc error: code = NotFound desc = could not find container \"1e5659f43eee7a0d449a6c665a4f99636ab1cef7504bc6dbe1e592640e45314f\": container with ID starting with 1e5659f43eee7a0d449a6c665a4f99636ab1cef7504bc6dbe1e592640e45314f not found: ID does not exist"
Oct 30 22:26:25.688849 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:25.688811    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn"]
Oct 30 22:26:25.743368 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:25.743326    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn"]
Oct 30 22:26:25.962876 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:25.962836    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="689c9d7f-c8ab-49e5-b647-b4a62d97c1e7" path="/var/lib/kubelet/pods/689c9d7f-c8ab-49e5-b647-b4a62d97c1e7/volumes"
Oct 30 22:26:25.963324 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:25.963308    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aec98483-f8d7-4eb3-811e-5f6b86e56230" path="/var/lib/kubelet/pods/aec98483-f8d7-4eb3-811e-5f6b86e56230/volumes"
Oct 30 22:26:26.091361 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:26.091328    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:26.306920 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:26.306819    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:26.464937 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:26.464885    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:27.043407 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:27.043373    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:27.668786 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:27.668730    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63a282-75688d8f4b-sthqn" podUID="aec98483-f8d7-4eb3-811e-5f6b86e56230" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.157:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:26:28.170522 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:28.170474    4871 patch_prober.go:29] interesting pod/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:26:28.170934 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:28.170557    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8" podUID="1d075c21-db31-400b-b173-02391280ad9f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:26:28.871654 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:28.871621    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"
Oct 30 22:26:28.988965 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:28.988927    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1d075c21-db31-400b-b173-02391280ad9f-istio-token\" (UniqueName: \"kubernetes.io/projected/1d075c21-db31-400b-b173-02391280ad9f-istio-token\") pod \"1d075c21-db31-400b-b173-02391280ad9f\" (UID: \"1d075c21-db31-400b-b173-02391280ad9f\") "
Oct 30 22:26:28.989126 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:28.989010    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1d075c21-db31-400b-b173-02391280ad9f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d075c21-db31-400b-b173-02391280ad9f-istiod-ca-cert\") pod \"1d075c21-db31-400b-b173-02391280ad9f\" (UID: \"1d075c21-db31-400b-b173-02391280ad9f\") "
Oct 30 22:26:28.989126 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:28.989048    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1d075c21-db31-400b-b173-02391280ad9f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d075c21-db31-400b-b173-02391280ad9f-istio-podinfo\") pod \"1d075c21-db31-400b-b173-02391280ad9f\" (UID: \"1d075c21-db31-400b-b173-02391280ad9f\") "
Oct 30 22:26:28.989126 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:28.989095    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d075c21-db31-400b-b173-02391280ad9f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d075c21-db31-400b-b173-02391280ad9f-workload-socket\") pod \"1d075c21-db31-400b-b173-02391280ad9f\" (UID: \"1d075c21-db31-400b-b173-02391280ad9f\") "
Oct 30 22:26:28.989335 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:28.989163    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d075c21-db31-400b-b173-02391280ad9f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d075c21-db31-400b-b173-02391280ad9f-workload-certs\") pod \"1d075c21-db31-400b-b173-02391280ad9f\" (UID: \"1d075c21-db31-400b-b173-02391280ad9f\") "
Oct 30 22:26:28.989335 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:28.989272    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d075c21-db31-400b-b173-02391280ad9f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d075c21-db31-400b-b173-02391280ad9f-istio-envoy\") pod \"1d075c21-db31-400b-b173-02391280ad9f\" (UID: \"1d075c21-db31-400b-b173-02391280ad9f\") "
Oct 30 22:26:28.989452 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:28.989348    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d075c21-db31-400b-b173-02391280ad9f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d075c21-db31-400b-b173-02391280ad9f-credential-socket\") pod \"1d075c21-db31-400b-b173-02391280ad9f\" (UID: \"1d075c21-db31-400b-b173-02391280ad9f\") "
Oct 30 22:26:28.989452 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:28.989425    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1d075c21-db31-400b-b173-02391280ad9f-kube-api-access-xgzvm\" (UniqueName: \"kubernetes.io/projected/1d075c21-db31-400b-b173-02391280ad9f-kube-api-access-xgzvm\") pod \"1d075c21-db31-400b-b173-02391280ad9f\" (UID: \"1d075c21-db31-400b-b173-02391280ad9f\") "
Oct 30 22:26:28.989577 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:28.989453    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d075c21-db31-400b-b173-02391280ad9f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d075c21-db31-400b-b173-02391280ad9f-istio-data\") pod \"1d075c21-db31-400b-b173-02391280ad9f\" (UID: \"1d075c21-db31-400b-b173-02391280ad9f\") "
Oct 30 22:26:28.989639 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:28.989563    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d075c21-db31-400b-b173-02391280ad9f-workload-certs" pod "1d075c21-db31-400b-b173-02391280ad9f" (UID: "1d075c21-db31-400b-b173-02391280ad9f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:28.989639 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:28.989579    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d075c21-db31-400b-b173-02391280ad9f-workload-socket" pod "1d075c21-db31-400b-b173-02391280ad9f" (UID: "1d075c21-db31-400b-b173-02391280ad9f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:28.989639 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:28.989575    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d075c21-db31-400b-b173-02391280ad9f-istiod-ca-cert" pod "1d075c21-db31-400b-b173-02391280ad9f" (UID: "1d075c21-db31-400b-b173-02391280ad9f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:26:28.989824 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:28.989794    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d075c21-db31-400b-b173-02391280ad9f-istio-data" pod "1d075c21-db31-400b-b173-02391280ad9f" (UID: "1d075c21-db31-400b-b173-02391280ad9f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:28.989949 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:28.989923    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d075c21-db31-400b-b173-02391280ad9f-credential-socket" pod "1d075c21-db31-400b-b173-02391280ad9f" (UID: "1d075c21-db31-400b-b173-02391280ad9f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:28.990096 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:28.990074    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d075c21-db31-400b-b173-02391280ad9f-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:28.990172 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:28.990103    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d075c21-db31-400b-b173-02391280ad9f-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:28.990172 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:28.990120    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d075c21-db31-400b-b173-02391280ad9f-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:28.990172 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:28.990135    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d075c21-db31-400b-b173-02391280ad9f-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:28.990172 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:28.990149    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d075c21-db31-400b-b173-02391280ad9f-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:28.991481 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:28.991453    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1d075c21-db31-400b-b173-02391280ad9f-istio-podinfo" pod "1d075c21-db31-400b-b173-02391280ad9f" (UID: "1d075c21-db31-400b-b173-02391280ad9f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:26:28.991694 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:28.991670    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d075c21-db31-400b-b173-02391280ad9f-istio-envoy" pod "1d075c21-db31-400b-b173-02391280ad9f" (UID: "1d075c21-db31-400b-b173-02391280ad9f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:28.991694 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:28.991685    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d075c21-db31-400b-b173-02391280ad9f-kube-api-access-xgzvm" pod "1d075c21-db31-400b-b173-02391280ad9f" (UID: "1d075c21-db31-400b-b173-02391280ad9f"). InnerVolumeSpecName "kube-api-access-xgzvm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:26:28.991824 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:28.991717    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d075c21-db31-400b-b173-02391280ad9f-istio-token" pod "1d075c21-db31-400b-b173-02391280ad9f" (UID: "1d075c21-db31-400b-b173-02391280ad9f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:26:29.091528 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:29.091492    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d075c21-db31-400b-b173-02391280ad9f-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:29.091528 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:29.091531    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgzvm\" (UniqueName: \"kubernetes.io/projected/1d075c21-db31-400b-b173-02391280ad9f-kube-api-access-xgzvm\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:29.091774 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:29.091547    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d075c21-db31-400b-b173-02391280ad9f-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:29.091774 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:29.091560    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d075c21-db31-400b-b173-02391280ad9f-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:29.552579 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:29.552544    4871 generic.go:356] "Generic (PLEG): container finished" podID="1d075c21-db31-400b-b173-02391280ad9f" containerID="3df5dfb85d51185b3f122c9e766848737c76dc07154a01a0fb0f093f72a62067" exitCode=0
Oct 30 22:26:29.553014 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:29.552599    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"
Oct 30 22:26:29.553014 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:29.552636    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8" event={"ID":"1d075c21-db31-400b-b173-02391280ad9f","Type":"ContainerDied","Data":"3df5dfb85d51185b3f122c9e766848737c76dc07154a01a0fb0f093f72a62067"}
Oct 30 22:26:29.553014 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:29.552693    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8" event={"ID":"1d075c21-db31-400b-b173-02391280ad9f","Type":"ContainerDied","Data":"bbaf6e7d72ae8504527a3b44b9718f87baeb9874583403aec785bbfaffff21b2"}
Oct 30 22:26:29.553014 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:29.552725    4871 scope.go:122] "RemoveContainer" containerID="3df5dfb85d51185b3f122c9e766848737c76dc07154a01a0fb0f093f72a62067"
Oct 30 22:26:29.565498 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:29.565476    4871 scope.go:122] "RemoveContainer" containerID="c1115d043b1b29267fd73e98f53673914840a539e9549af2910422ba997edb30"
Oct 30 22:26:29.575971 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:29.575950    4871 scope.go:122] "RemoveContainer" containerID="3df5dfb85d51185b3f122c9e766848737c76dc07154a01a0fb0f093f72a62067"
Oct 30 22:26:29.576299 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:26:29.576281    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3df5dfb85d51185b3f122c9e766848737c76dc07154a01a0fb0f093f72a62067\": container with ID starting with 3df5dfb85d51185b3f122c9e766848737c76dc07154a01a0fb0f093f72a62067 not found: ID does not exist" containerID="3df5dfb85d51185b3f122c9e766848737c76dc07154a01a0fb0f093f72a62067"
Oct 30 22:26:29.576369 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:29.576310    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3df5dfb85d51185b3f122c9e766848737c76dc07154a01a0fb0f093f72a62067"} err="failed to get container status \"3df5dfb85d51185b3f122c9e766848737c76dc07154a01a0fb0f093f72a62067\": rpc error: code = NotFound desc = could not find container \"3df5dfb85d51185b3f122c9e766848737c76dc07154a01a0fb0f093f72a62067\": container with ID starting with 3df5dfb85d51185b3f122c9e766848737c76dc07154a01a0fb0f093f72a62067 not found: ID does not exist"
Oct 30 22:26:29.576369 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:29.576329    4871 scope.go:122] "RemoveContainer" containerID="c1115d043b1b29267fd73e98f53673914840a539e9549af2910422ba997edb30"
Oct 30 22:26:29.576617 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:26:29.576593    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1115d043b1b29267fd73e98f53673914840a539e9549af2910422ba997edb30\": container with ID starting with c1115d043b1b29267fd73e98f53673914840a539e9549af2910422ba997edb30 not found: ID does not exist" containerID="c1115d043b1b29267fd73e98f53673914840a539e9549af2910422ba997edb30"
Oct 30 22:26:29.576675 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:29.576627    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1115d043b1b29267fd73e98f53673914840a539e9549af2910422ba997edb30"} err="failed to get container status \"c1115d043b1b29267fd73e98f53673914840a539e9549af2910422ba997edb30\": rpc error: code = NotFound desc = could not find container \"c1115d043b1b29267fd73e98f53673914840a539e9549af2910422ba997edb30\": container with ID starting with c1115d043b1b29267fd73e98f53673914840a539e9549af2910422ba997edb30 not found: ID does not exist"
Oct 30 22:26:29.726679 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:29.726637    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"]
Oct 30 22:26:29.778647 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:29.778613    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-ct8l8"]
Oct 30 22:26:29.960451 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:29.960409    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1d075c21-db31-400b-b173-02391280ad9f" path="/var/lib/kubelet/pods/1d075c21-db31-400b-b173-02391280ad9f/volumes"
Oct 30 22:26:31.205776 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:31.205723    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:32.084533 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:32.084498    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:35.570573 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:26:35.570489    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:26:35.915590 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:35.915550    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:42.091537 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:42.091500    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:45.289344 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:45.289304    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:45.446810 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:45.446774    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:45.606352 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:45.606229    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:46.757532 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:46.757490    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:46.913205 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:46.913169    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:51.325819 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:51.325781    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:55.842134 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:26:55.842100    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:00.515182 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:00.515148    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:05.137413 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:05.137376    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:05.572080 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:27:05.571993    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:27:08.645348 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:08.645304    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:09.542715 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:09.542681    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:09.837986 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:09.837894    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:14.321300 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:14.321265    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:15.834400 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:15.834358    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:16.028663 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:16.028618    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:16.249115 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:16.249072    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"]
Oct 30 22:27:16.249340 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:16.249310    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2" podUID="cf4abc58-fe24-486b-a195-1eb2a3b2e643" containerName="istio-proxy" containerID="cri-o://19afab03874a37cb1c0466fa9cecc8a14a2b2e90573756821cfa903f027967a1" gracePeriod=30
Oct 30 22:27:16.269589 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:16.269509    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:19.107377 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:19.107334    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:21.518087 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:21.518011    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"
Oct 30 22:27:21.626980 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:21.626934    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf4abc58-fe24-486b-a195-1eb2a3b2e643-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf4abc58-fe24-486b-a195-1eb2a3b2e643-istio-data\") pod \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\" (UID: \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\") "
Oct 30 22:27:21.627198 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:21.627007    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf4abc58-fe24-486b-a195-1eb2a3b2e643-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf4abc58-fe24-486b-a195-1eb2a3b2e643-credential-socket\") pod \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\" (UID: \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\") "
Oct 30 22:27:21.627198 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:21.627078    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf4abc58-fe24-486b-a195-1eb2a3b2e643-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf4abc58-fe24-486b-a195-1eb2a3b2e643-workload-socket\") pod \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\" (UID: \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\") "
Oct 30 22:27:21.627198 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:21.627124    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf4abc58-fe24-486b-a195-1eb2a3b2e643-kube-api-access-rgb48\" (UniqueName: \"kubernetes.io/projected/cf4abc58-fe24-486b-a195-1eb2a3b2e643-kube-api-access-rgb48\") pod \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\" (UID: \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\") "
Oct 30 22:27:21.627198 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:21.627171    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf4abc58-fe24-486b-a195-1eb2a3b2e643-istio-token\" (UniqueName: \"kubernetes.io/projected/cf4abc58-fe24-486b-a195-1eb2a3b2e643-istio-token\") pod \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\" (UID: \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\") "
Oct 30 22:27:21.627475 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:21.627218    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cf4abc58-fe24-486b-a195-1eb2a3b2e643-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf4abc58-fe24-486b-a195-1eb2a3b2e643-istiod-ca-cert\") pod \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\" (UID: \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\") "
Oct 30 22:27:21.627475 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:21.627281    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf4abc58-fe24-486b-a195-1eb2a3b2e643-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf4abc58-fe24-486b-a195-1eb2a3b2e643-workload-certs\") pod \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\" (UID: \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\") "
Oct 30 22:27:21.627475 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:21.627314    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cf4abc58-fe24-486b-a195-1eb2a3b2e643-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf4abc58-fe24-486b-a195-1eb2a3b2e643-istio-podinfo\") pod \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\" (UID: \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\") "
Oct 30 22:27:21.627475 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:21.627323    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf4abc58-fe24-486b-a195-1eb2a3b2e643-istio-data" pod "cf4abc58-fe24-486b-a195-1eb2a3b2e643" (UID: "cf4abc58-fe24-486b-a195-1eb2a3b2e643"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:27:21.627475 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:21.627364    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf4abc58-fe24-486b-a195-1eb2a3b2e643-credential-socket" pod "cf4abc58-fe24-486b-a195-1eb2a3b2e643" (UID: "cf4abc58-fe24-486b-a195-1eb2a3b2e643"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:27:21.627475 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:21.627387    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf4abc58-fe24-486b-a195-1eb2a3b2e643-workload-socket" pod "cf4abc58-fe24-486b-a195-1eb2a3b2e643" (UID: "cf4abc58-fe24-486b-a195-1eb2a3b2e643"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:27:21.627475 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:21.627400    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf4abc58-fe24-486b-a195-1eb2a3b2e643-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf4abc58-fe24-486b-a195-1eb2a3b2e643-istio-envoy\") pod \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\" (UID: \"cf4abc58-fe24-486b-a195-1eb2a3b2e643\") "
Oct 30 22:27:21.627859 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:21.627602    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf4abc58-fe24-486b-a195-1eb2a3b2e643-workload-certs" pod "cf4abc58-fe24-486b-a195-1eb2a3b2e643" (UID: "cf4abc58-fe24-486b-a195-1eb2a3b2e643"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:27:21.627859 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:21.627701    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf4abc58-fe24-486b-a195-1eb2a3b2e643-istiod-ca-cert" pod "cf4abc58-fe24-486b-a195-1eb2a3b2e643" (UID: "cf4abc58-fe24-486b-a195-1eb2a3b2e643"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:27:21.627988 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:21.627864    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf4abc58-fe24-486b-a195-1eb2a3b2e643-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:27:21.627988 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:21.627883    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf4abc58-fe24-486b-a195-1eb2a3b2e643-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:27:21.627988 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:21.627898    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf4abc58-fe24-486b-a195-1eb2a3b2e643-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:27:21.627988 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:21.627912    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf4abc58-fe24-486b-a195-1eb2a3b2e643-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:27:21.627988 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:21.627928    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf4abc58-fe24-486b-a195-1eb2a3b2e643-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:27:21.629582 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:21.629557    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cf4abc58-fe24-486b-a195-1eb2a3b2e643-istio-podinfo" pod "cf4abc58-fe24-486b-a195-1eb2a3b2e643" (UID: "cf4abc58-fe24-486b-a195-1eb2a3b2e643"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:27:21.629952 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:21.629927    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf4abc58-fe24-486b-a195-1eb2a3b2e643-istio-envoy" pod "cf4abc58-fe24-486b-a195-1eb2a3b2e643" (UID: "cf4abc58-fe24-486b-a195-1eb2a3b2e643"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:27:21.630035 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:21.629960    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf4abc58-fe24-486b-a195-1eb2a3b2e643-kube-api-access-rgb48" pod "cf4abc58-fe24-486b-a195-1eb2a3b2e643" (UID: "cf4abc58-fe24-486b-a195-1eb2a3b2e643"). InnerVolumeSpecName "kube-api-access-rgb48". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:27:21.630035 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:21.629979    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf4abc58-fe24-486b-a195-1eb2a3b2e643-istio-token" pod "cf4abc58-fe24-486b-a195-1eb2a3b2e643" (UID: "cf4abc58-fe24-486b-a195-1eb2a3b2e643"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:27:21.729188 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:21.729142    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rgb48\" (UniqueName: \"kubernetes.io/projected/cf4abc58-fe24-486b-a195-1eb2a3b2e643-kube-api-access-rgb48\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:27:21.729188 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:21.729183    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf4abc58-fe24-486b-a195-1eb2a3b2e643-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:27:21.729419 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:21.729197    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf4abc58-fe24-486b-a195-1eb2a3b2e643-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:27:21.729419 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:21.729211    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf4abc58-fe24-486b-a195-1eb2a3b2e643-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:27:21.944261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:21.944197    4871 generic.go:356] "Generic (PLEG): container finished" podID="cf4abc58-fe24-486b-a195-1eb2a3b2e643" containerID="19afab03874a37cb1c0466fa9cecc8a14a2b2e90573756821cfa903f027967a1" exitCode=0
Oct 30 22:27:21.944457 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:21.944274    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"
Oct 30 22:27:21.944457 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:21.944289    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2" event={"ID":"cf4abc58-fe24-486b-a195-1eb2a3b2e643","Type":"ContainerDied","Data":"19afab03874a37cb1c0466fa9cecc8a14a2b2e90573756821cfa903f027967a1"}
Oct 30 22:27:21.944457 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:21.944344    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2" event={"ID":"cf4abc58-fe24-486b-a195-1eb2a3b2e643","Type":"ContainerDied","Data":"dd32447f7f0073cfea07ebf84049a6a102f06f307320c738f8549dfb34e14729"}
Oct 30 22:27:21.944457 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:21.944366    4871 scope.go:122] "RemoveContainer" containerID="19afab03874a37cb1c0466fa9cecc8a14a2b2e90573756821cfa903f027967a1"
Oct 30 22:27:21.956064 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:21.956042    4871 scope.go:122] "RemoveContainer" containerID="0cbc0d71509db4413284c81b3a059c69a31415cd63a8b165fe83724ac10162f9"
Oct 30 22:27:21.966634 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:21.966614    4871 scope.go:122] "RemoveContainer" containerID="19afab03874a37cb1c0466fa9cecc8a14a2b2e90573756821cfa903f027967a1"
Oct 30 22:27:21.966955 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:27:21.966928    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19afab03874a37cb1c0466fa9cecc8a14a2b2e90573756821cfa903f027967a1\": container with ID starting with 19afab03874a37cb1c0466fa9cecc8a14a2b2e90573756821cfa903f027967a1 not found: ID does not exist" containerID="19afab03874a37cb1c0466fa9cecc8a14a2b2e90573756821cfa903f027967a1"
Oct 30 22:27:21.967075 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:21.966960    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19afab03874a37cb1c0466fa9cecc8a14a2b2e90573756821cfa903f027967a1"} err="failed to get container status \"19afab03874a37cb1c0466fa9cecc8a14a2b2e90573756821cfa903f027967a1\": rpc error: code = NotFound desc = could not find container \"19afab03874a37cb1c0466fa9cecc8a14a2b2e90573756821cfa903f027967a1\": container with ID starting with 19afab03874a37cb1c0466fa9cecc8a14a2b2e90573756821cfa903f027967a1 not found: ID does not exist"
Oct 30 22:27:21.967075 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:21.966976    4871 scope.go:122] "RemoveContainer" containerID="0cbc0d71509db4413284c81b3a059c69a31415cd63a8b165fe83724ac10162f9"
Oct 30 22:27:21.967280 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:27:21.967257    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cbc0d71509db4413284c81b3a059c69a31415cd63a8b165fe83724ac10162f9\": container with ID starting with 0cbc0d71509db4413284c81b3a059c69a31415cd63a8b165fe83724ac10162f9 not found: ID does not exist" containerID="0cbc0d71509db4413284c81b3a059c69a31415cd63a8b165fe83724ac10162f9"
Oct 30 22:27:21.967336 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:21.967289    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cbc0d71509db4413284c81b3a059c69a31415cd63a8b165fe83724ac10162f9"} err="failed to get container status \"0cbc0d71509db4413284c81b3a059c69a31415cd63a8b165fe83724ac10162f9\": rpc error: code = NotFound desc = could not find container \"0cbc0d71509db4413284c81b3a059c69a31415cd63a8b165fe83724ac10162f9\": container with ID starting with 0cbc0d71509db4413284c81b3a059c69a31415cd63a8b165fe83724ac10162f9 not found: ID does not exist"
Oct 30 22:27:22.119026 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:22.118990    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"]
Oct 30 22:27:22.170932 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:22.170902    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-63a282-openshift-default-7b6c7d9667-79fv2"]
Oct 30 22:27:23.961761 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:23.961724    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cf4abc58-fe24-486b-a195-1eb2a3b2e643" path="/var/lib/kubelet/pods/cf4abc58-fe24-486b-a195-1eb2a3b2e643/volumes"
Oct 30 22:27:25.293788 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:25.293753    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:28.231152 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:28.231115    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:29.913331 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:29.913285    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:34.576682 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:34.576639    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:35.569010 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:27:35.568971    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:27:38.576660 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:38.576630    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx_8d401654-8a4c-4a79-b2d3-612765312421/extract/0.log"
Oct 30 22:27:38.581610 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:38.581580    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx_8d401654-8a4c-4a79-b2d3-612765312421/extract/0.log"
Oct 30 22:27:38.591044 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:38.591021    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/2.log"
Oct 30 22:27:38.595827 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:38.595805    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/2.log"
Oct 30 22:27:38.599818 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:38.599794    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzntd_2360fddc-52a5-42a5-82ca-a05aff84d217/ovn-acl-logging/0.log"
Oct 30 22:27:38.603858 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:38.603836    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzntd_2360fddc-52a5-42a5-82ca-a05aff84d217/ovn-acl-logging/0.log"
Oct 30 22:27:39.178477 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:39.178433    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:43.918574 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:43.918530    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:45.803597 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:45.803560    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:47.260541 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:47.260506    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:49.094071 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:49.093978    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:53.705679 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:53.705649    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:57.104461 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:57.104427    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:57.300801 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:57.300764    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:58.323129 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:27:58.323095    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:02.395215 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:02.395171    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:02.942965 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:02.942927    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:03.225255 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:03.225139    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:04.006696 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.006647    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:04.259474 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.259381    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:04.542334 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.542203    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:04.764265 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.764201    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"]
Oct 30 22:28:04.764820 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.764795    4871 cpu_manager.go:414] "containerMap: removing container" podUID="689c9d7f-c8ab-49e5-b647-b4a62d97c1e7" containerName="istio-proxy"
Oct 30 22:28:04.764820 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.764819    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="689c9d7f-c8ab-49e5-b647-b4a62d97c1e7" containerName="istio-proxy"
Oct 30 22:28:04.765006 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.764841    4871 cpu_manager.go:414] "containerMap: removing container" podUID="cf4abc58-fe24-486b-a195-1eb2a3b2e643" containerName="istio-proxy"
Oct 30 22:28:04.765006 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.764849    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf4abc58-fe24-486b-a195-1eb2a3b2e643" containerName="istio-proxy"
Oct 30 22:28:04.765006 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.764860    4871 cpu_manager.go:414] "containerMap: removing container" podUID="1fad116b-e1cc-42da-a439-bdecb46b96a9" containerName="istio-proxy"
Oct 30 22:28:04.765006 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.764868    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1fad116b-e1cc-42da-a439-bdecb46b96a9" containerName="istio-proxy"
Oct 30 22:28:04.765006 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.764902    4871 cpu_manager.go:414] "containerMap: removing container" podUID="4d334339-1d4b-421b-856e-c2a0d9f4737a" containerName="istio-proxy"
Oct 30 22:28:04.765006 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.764909    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4d334339-1d4b-421b-856e-c2a0d9f4737a" containerName="istio-proxy"
Oct 30 22:28:04.765006 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.764921    4871 cpu_manager.go:414] "containerMap: removing container" podUID="d6407e4e-8850-4c67-8ad5-93c7fde3ea55" containerName="istio-proxy"
Oct 30 22:28:04.765006 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.764928    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d6407e4e-8850-4c67-8ad5-93c7fde3ea55" containerName="istio-proxy"
Oct 30 22:28:04.765006 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.764936    4871 cpu_manager.go:414] "containerMap: removing container" podUID="c2b31083-b29f-4a68-a20c-853287a23d6e" containerName="maas-api"
Oct 30 22:28:04.765006 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.764943    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c2b31083-b29f-4a68-a20c-853287a23d6e" containerName="maas-api"
Oct 30 22:28:04.765006 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.764956    4871 cpu_manager.go:414] "containerMap: removing container" podUID="4d334339-1d4b-421b-856e-c2a0d9f4737a" containerName="istio-proxy"
Oct 30 22:28:04.765006 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.764964    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4d334339-1d4b-421b-856e-c2a0d9f4737a" containerName="istio-proxy"
Oct 30 22:28:04.765006 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.764977    4871 cpu_manager.go:414] "containerMap: removing container" podUID="1d075c21-db31-400b-b173-02391280ad9f" containerName="istio-proxy"
Oct 30 22:28:04.765006 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.764984    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1d075c21-db31-400b-b173-02391280ad9f" containerName="istio-proxy"
Oct 30 22:28:04.765006 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.764993    4871 cpu_manager.go:414] "containerMap: removing container" podUID="cf4abc58-fe24-486b-a195-1eb2a3b2e643" containerName="istio-proxy"
Oct 30 22:28:04.765006 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.765000    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf4abc58-fe24-486b-a195-1eb2a3b2e643" containerName="istio-proxy"
Oct 30 22:28:04.765006 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.765012    4871 cpu_manager.go:414] "containerMap: removing container" podUID="5d03255e-3722-45d9-b21b-b1c751355173" containerName="maas-api"
Oct 30 22:28:04.765787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.765019    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5d03255e-3722-45d9-b21b-b1c751355173" containerName="maas-api"
Oct 30 22:28:04.765787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.765032    4871 cpu_manager.go:414] "containerMap: removing container" podUID="914386df-d6fd-4b9d-92b9-45a987e3b514" containerName="payload-processing"
Oct 30 22:28:04.765787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.765039    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="914386df-d6fd-4b9d-92b9-45a987e3b514" containerName="payload-processing"
Oct 30 22:28:04.765787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.765049    4871 cpu_manager.go:414] "containerMap: removing container" podUID="dd4e1319-f9e3-46e5-856b-67e2f5139b6e" containerName="istio-proxy"
Oct 30 22:28:04.765787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.765056    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dd4e1319-f9e3-46e5-856b-67e2f5139b6e" containerName="istio-proxy"
Oct 30 22:28:04.765787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.765076    4871 cpu_manager.go:414] "containerMap: removing container" podUID="dd4e1319-f9e3-46e5-856b-67e2f5139b6e" containerName="istio-proxy"
Oct 30 22:28:04.765787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.765083    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dd4e1319-f9e3-46e5-856b-67e2f5139b6e" containerName="istio-proxy"
Oct 30 22:28:04.765787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.765094    4871 cpu_manager.go:414] "containerMap: removing container" podUID="1d075c21-db31-400b-b173-02391280ad9f" containerName="istio-proxy"
Oct 30 22:28:04.765787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.765100    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1d075c21-db31-400b-b173-02391280ad9f" containerName="istio-proxy"
Oct 30 22:28:04.765787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.765110    4871 cpu_manager.go:414] "containerMap: removing container" podUID="d6407e4e-8850-4c67-8ad5-93c7fde3ea55" containerName="istio-proxy"
Oct 30 22:28:04.765787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.765117    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d6407e4e-8850-4c67-8ad5-93c7fde3ea55" containerName="istio-proxy"
Oct 30 22:28:04.765787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.765129    4871 cpu_manager.go:414] "containerMap: removing container" podUID="aec98483-f8d7-4eb3-811e-5f6b86e56230" containerName="maas-api"
Oct 30 22:28:04.765787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.765136    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aec98483-f8d7-4eb3-811e-5f6b86e56230" containerName="maas-api"
Oct 30 22:28:04.765787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.765145    4871 cpu_manager.go:414] "containerMap: removing container" podUID="53120b48-0737-4956-965b-bf544e8c1096" containerName="maas-api"
Oct 30 22:28:04.765787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.765152    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="53120b48-0737-4956-965b-bf544e8c1096" containerName="maas-api"
Oct 30 22:28:04.765787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.765303    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="1d075c21-db31-400b-b173-02391280ad9f" containerName="istio-proxy"
Oct 30 22:28:04.765787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.765319    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf4abc58-fe24-486b-a195-1eb2a3b2e643" containerName="istio-proxy"
Oct 30 22:28:04.765787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.765327    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="5d03255e-3722-45d9-b21b-b1c751355173" containerName="maas-api"
Oct 30 22:28:04.765787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.765338    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="914386df-d6fd-4b9d-92b9-45a987e3b514" containerName="payload-processing"
Oct 30 22:28:04.765787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.765348    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="dd4e1319-f9e3-46e5-856b-67e2f5139b6e" containerName="istio-proxy"
Oct 30 22:28:04.765787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.765357    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="dd4e1319-f9e3-46e5-856b-67e2f5139b6e" containerName="istio-proxy"
Oct 30 22:28:04.765787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.765367    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="1d075c21-db31-400b-b173-02391280ad9f" containerName="istio-proxy"
Oct 30 22:28:04.765787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.765375    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="d6407e4e-8850-4c67-8ad5-93c7fde3ea55" containerName="istio-proxy"
Oct 30 22:28:04.765787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.765384    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="aec98483-f8d7-4eb3-811e-5f6b86e56230" containerName="maas-api"
Oct 30 22:28:04.765787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.765391    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="53120b48-0737-4956-965b-bf544e8c1096" containerName="maas-api"
Oct 30 22:28:04.765787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.765400    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="1fad116b-e1cc-42da-a439-bdecb46b96a9" containerName="istio-proxy"
Oct 30 22:28:04.765787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.765410    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="689c9d7f-c8ab-49e5-b647-b4a62d97c1e7" containerName="istio-proxy"
Oct 30 22:28:04.765787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.765420    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="1fad116b-e1cc-42da-a439-bdecb46b96a9" containerName="istio-proxy"
Oct 30 22:28:04.765787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.765432    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="689c9d7f-c8ab-49e5-b647-b4a62d97c1e7" containerName="istio-proxy"
Oct 30 22:28:04.765787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.765450    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="4d334339-1d4b-421b-856e-c2a0d9f4737a" containerName="istio-proxy"
Oct 30 22:28:04.765787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.765459    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="c2b31083-b29f-4a68-a20c-853287a23d6e" containerName="maas-api"
Oct 30 22:28:04.765787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.765469    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="4d334339-1d4b-421b-856e-c2a0d9f4737a" containerName="istio-proxy"
Oct 30 22:28:04.770466 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.770445    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"
Oct 30 22:28:04.832183 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.832081    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-istio-data\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n\" (UID: \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"
Oct 30 22:28:04.832183 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.832176    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-istio-envoy\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n\" (UID: \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"
Oct 30 22:28:04.832455 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.832213    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-workload-socket\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n\" (UID: \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"
Oct 30 22:28:04.832455 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.832271    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgxfq\" (UniqueName: \"kubernetes.io/projected/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-kube-api-access-qgxfq\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n\" (UID: \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"
Oct 30 22:28:04.832455 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.832318    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-istio-podinfo\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n\" (UID: \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"
Oct 30 22:28:04.832455 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.832381    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-istiod-ca-cert\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n\" (UID: \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"
Oct 30 22:28:04.832455 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.832406    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-credential-socket\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n\" (UID: \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"
Oct 30 22:28:04.832455 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.832448    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-workload-certs\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n\" (UID: \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"
Oct 30 22:28:04.832813 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.832478    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-istio-token\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n\" (UID: \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"
Oct 30 22:28:04.832931 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.832904    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"]
Oct 30 22:28:04.933139 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.933092    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-workload-certs\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n\" (UID: \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"
Oct 30 22:28:04.933139 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.933134    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-istio-token\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n\" (UID: \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"
Oct 30 22:28:04.933460 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.933200    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-istio-data\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n\" (UID: \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"
Oct 30 22:28:04.933460 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.933277    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-istio-envoy\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n\" (UID: \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"
Oct 30 22:28:04.933460 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.933310    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-workload-socket\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n\" (UID: \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"
Oct 30 22:28:04.933460 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.933341    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgxfq\" (UniqueName: \"kubernetes.io/projected/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-kube-api-access-qgxfq\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n\" (UID: \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"
Oct 30 22:28:04.933460 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.933377    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-istio-podinfo\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n\" (UID: \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"
Oct 30 22:28:04.933460 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.933436    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-istiod-ca-cert\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n\" (UID: \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"
Oct 30 22:28:04.933773 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.933469    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-credential-socket\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n\" (UID: \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"
Oct 30 22:28:04.933773 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.933637    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-workload-certs\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n\" (UID: \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"
Oct 30 22:28:04.933884 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.933768    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-istio-data\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n\" (UID: \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"
Oct 30 22:28:04.933939 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.933888    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-workload-socket\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n\" (UID: \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"
Oct 30 22:28:04.934164 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.934144    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-credential-socket\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n\" (UID: \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"
Oct 30 22:28:04.934419 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.934394    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-istiod-ca-cert\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n\" (UID: \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"
Oct 30 22:28:04.935880 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.935849    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-istio-envoy\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n\" (UID: \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"
Oct 30 22:28:04.936125 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:04.936104    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-istio-podinfo\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n\" (UID: \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"
Oct 30 22:28:05.004440 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:05.004407    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-istio-token\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n\" (UID: \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"
Oct 30 22:28:05.006835 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:05.006803    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgxfq\" (UniqueName: \"kubernetes.io/projected/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-kube-api-access-qgxfq\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n\" (UID: \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"
Oct 30 22:28:05.084549 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:05.084450    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"
Oct 30 22:28:05.251190 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:05.251157    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:05.516041 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:28:05.516004    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40bab067_19f1_4ebc_b467_3f0f2e1bbf48.slice/crio-a6ca5cf9598297a3e23042b1511e5a9f4d2ab947ad62ed5efce0e52aacd9c8a0 WatchSource:0}: Error finding container a6ca5cf9598297a3e23042b1511e5a9f4d2ab947ad62ed5efce0e52aacd9c8a0: Status 404 returned error can't find the container with id a6ca5cf9598297a3e23042b1511e5a9f4d2ab947ad62ed5efce0e52aacd9c8a0
Oct 30 22:28:05.518621 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:05.518587    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:28:05.518703 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:05.518658    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:28:05.518703 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:05.518693    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:28:05.521581 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:05.521555    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"]
Oct 30 22:28:05.577925 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:28:05.577890    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:28:06.346813 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:06.341345    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n" event={"ID":"40bab067-19f1-4ebc-b467-3f0f2e1bbf48","Type":"ContainerStarted","Data":"806602b1d25a3284cbc3ceeb0b13abdc9f512f922adc6dbaab6ba3d037f5cbf3"}
Oct 30 22:28:06.346813 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:06.341467    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n" event={"ID":"40bab067-19f1-4ebc-b467-3f0f2e1bbf48","Type":"ContainerStarted","Data":"a6ca5cf9598297a3e23042b1511e5a9f4d2ab947ad62ed5efce0e52aacd9c8a0"}
Oct 30 22:28:06.564280 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:06.564199    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n" podStartSLOduration=2.56418276 podStartE2EDuration="2.56418276s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:28:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:28:06.533175073 +0000 UTC m=+3028.710663692" watchObservedRunningTime="2026-10-30 22:28:06.56418276 +0000 UTC m=+3028.741671376"
Oct 30 22:28:07.085093 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:07.085054    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"
Oct 30 22:28:07.086813 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:07.086786    4871 patch_prober.go:29] interesting pod/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused" start-of-body=
Oct 30 22:28:07.086925 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:07.086837    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n" podUID="40bab067-19f1-4ebc-b467-3f0f2e1bbf48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused"
Oct 30 22:28:08.085101 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:08.085065    4871 patch_prober.go:29] interesting pod/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused" start-of-body=
Oct 30 22:28:08.085571 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:08.085124    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n" podUID="40bab067-19f1-4ebc-b467-3f0f2e1bbf48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused"
Oct 30 22:28:09.084922 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:09.084881    4871 patch_prober.go:29] interesting pod/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused" start-of-body=
Oct 30 22:28:09.085114 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:09.084937    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n" podUID="40bab067-19f1-4ebc-b467-3f0f2e1bbf48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused"
Oct 30 22:28:09.270689 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:09.270652    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:10.085829 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:10.085792    4871 patch_prober.go:29] interesting pod/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused" start-of-body=
Oct 30 22:28:10.086059 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:10.085849    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n" podUID="40bab067-19f1-4ebc-b467-3f0f2e1bbf48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused"
Oct 30 22:28:11.085348 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:11.085309    4871 patch_prober.go:29] interesting pod/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused" start-of-body=
Oct 30 22:28:11.085810 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:11.085374    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n" podUID="40bab067-19f1-4ebc-b467-3f0f2e1bbf48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused"
Oct 30 22:28:11.600269 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:11.600200    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:11.769050 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:11.769013    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:11.975022 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:11.974986    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:12.089263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:12.085307    4871 patch_prober.go:29] interesting pod/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused" start-of-body=
Oct 30 22:28:12.089263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:12.085378    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n" podUID="40bab067-19f1-4ebc-b467-3f0f2e1bbf48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused"
Oct 30 22:28:13.089199 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:13.089164    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"
Oct 30 22:28:13.089652 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:13.089626    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"
Oct 30 22:28:13.090442 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:13.090425    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"
Oct 30 22:28:13.921805 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:13.921771    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:18.518456 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:18.518410    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:23.601414 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:23.601372    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:25.226767 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.226729    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"]
Oct 30 22:28:25.227308 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.227290    4871 cpu_manager.go:414] "containerMap: removing container" podUID="689c9d7f-c8ab-49e5-b647-b4a62d97c1e7" containerName="istio-proxy"
Oct 30 22:28:25.227354 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.227310    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="689c9d7f-c8ab-49e5-b647-b4a62d97c1e7" containerName="istio-proxy"
Oct 30 22:28:25.227354 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.227338    4871 cpu_manager.go:414] "containerMap: removing container" podUID="1fad116b-e1cc-42da-a439-bdecb46b96a9" containerName="istio-proxy"
Oct 30 22:28:25.227354 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.227344    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1fad116b-e1cc-42da-a439-bdecb46b96a9" containerName="istio-proxy"
Oct 30 22:28:25.227501 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.227491    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf4abc58-fe24-486b-a195-1eb2a3b2e643" containerName="istio-proxy"
Oct 30 22:28:25.227540 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.227505    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="d6407e4e-8850-4c67-8ad5-93c7fde3ea55" containerName="istio-proxy"
Oct 30 22:28:25.231190 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.231168    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"
Oct 30 22:28:25.286978 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.286940    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c647346d-70fe-456e-b81e-f3e90f7bbf17-workload-certs\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz\" (UID: \"c647346d-70fe-456e-b81e-f3e90f7bbf17\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"
Oct 30 22:28:25.287160 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.286997    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c647346d-70fe-456e-b81e-f3e90f7bbf17-workload-socket\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz\" (UID: \"c647346d-70fe-456e-b81e-f3e90f7bbf17\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"
Oct 30 22:28:25.287160 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.287076    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c647346d-70fe-456e-b81e-f3e90f7bbf17-credential-socket\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz\" (UID: \"c647346d-70fe-456e-b81e-f3e90f7bbf17\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"
Oct 30 22:28:25.287160 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.287126    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c647346d-70fe-456e-b81e-f3e90f7bbf17-istio-token\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz\" (UID: \"c647346d-70fe-456e-b81e-f3e90f7bbf17\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"
Oct 30 22:28:25.287291 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.287194    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lk6d\" (UniqueName: \"kubernetes.io/projected/c647346d-70fe-456e-b81e-f3e90f7bbf17-kube-api-access-6lk6d\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz\" (UID: \"c647346d-70fe-456e-b81e-f3e90f7bbf17\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"
Oct 30 22:28:25.287291 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.287271    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c647346d-70fe-456e-b81e-f3e90f7bbf17-istio-envoy\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz\" (UID: \"c647346d-70fe-456e-b81e-f3e90f7bbf17\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"
Oct 30 22:28:25.287361 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.287334    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c647346d-70fe-456e-b81e-f3e90f7bbf17-istio-data\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz\" (UID: \"c647346d-70fe-456e-b81e-f3e90f7bbf17\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"
Oct 30 22:28:25.287397 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.287372    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c647346d-70fe-456e-b81e-f3e90f7bbf17-istio-podinfo\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz\" (UID: \"c647346d-70fe-456e-b81e-f3e90f7bbf17\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"
Oct 30 22:28:25.287432 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.287401    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c647346d-70fe-456e-b81e-f3e90f7bbf17-istiod-ca-cert\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz\" (UID: \"c647346d-70fe-456e-b81e-f3e90f7bbf17\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"
Oct 30 22:28:25.342017 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.341972    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"]
Oct 30 22:28:25.388848 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.388810    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c647346d-70fe-456e-b81e-f3e90f7bbf17-workload-certs\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz\" (UID: \"c647346d-70fe-456e-b81e-f3e90f7bbf17\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"
Oct 30 22:28:25.389073 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.388882    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c647346d-70fe-456e-b81e-f3e90f7bbf17-workload-socket\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz\" (UID: \"c647346d-70fe-456e-b81e-f3e90f7bbf17\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"
Oct 30 22:28:25.389073 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.388921    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c647346d-70fe-456e-b81e-f3e90f7bbf17-credential-socket\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz\" (UID: \"c647346d-70fe-456e-b81e-f3e90f7bbf17\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"
Oct 30 22:28:25.389073 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.388973    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c647346d-70fe-456e-b81e-f3e90f7bbf17-istio-token\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz\" (UID: \"c647346d-70fe-456e-b81e-f3e90f7bbf17\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"
Oct 30 22:28:25.389073 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.389015    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lk6d\" (UniqueName: \"kubernetes.io/projected/c647346d-70fe-456e-b81e-f3e90f7bbf17-kube-api-access-6lk6d\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz\" (UID: \"c647346d-70fe-456e-b81e-f3e90f7bbf17\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"
Oct 30 22:28:25.389073 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.389046    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c647346d-70fe-456e-b81e-f3e90f7bbf17-istio-envoy\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz\" (UID: \"c647346d-70fe-456e-b81e-f3e90f7bbf17\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"
Oct 30 22:28:25.389419 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.389090    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c647346d-70fe-456e-b81e-f3e90f7bbf17-istio-data\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz\" (UID: \"c647346d-70fe-456e-b81e-f3e90f7bbf17\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"
Oct 30 22:28:25.389419 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.389146    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c647346d-70fe-456e-b81e-f3e90f7bbf17-istio-podinfo\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz\" (UID: \"c647346d-70fe-456e-b81e-f3e90f7bbf17\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"
Oct 30 22:28:25.389419 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.389184    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c647346d-70fe-456e-b81e-f3e90f7bbf17-istiod-ca-cert\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz\" (UID: \"c647346d-70fe-456e-b81e-f3e90f7bbf17\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"
Oct 30 22:28:25.389419 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.389394    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c647346d-70fe-456e-b81e-f3e90f7bbf17-workload-certs\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz\" (UID: \"c647346d-70fe-456e-b81e-f3e90f7bbf17\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"
Oct 30 22:28:25.389644 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.389534    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c647346d-70fe-456e-b81e-f3e90f7bbf17-credential-socket\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz\" (UID: \"c647346d-70fe-456e-b81e-f3e90f7bbf17\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"
Oct 30 22:28:25.389644 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.389598    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c647346d-70fe-456e-b81e-f3e90f7bbf17-istio-data\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz\" (UID: \"c647346d-70fe-456e-b81e-f3e90f7bbf17\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"
Oct 30 22:28:25.389757 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.389652    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c647346d-70fe-456e-b81e-f3e90f7bbf17-workload-socket\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz\" (UID: \"c647346d-70fe-456e-b81e-f3e90f7bbf17\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"
Oct 30 22:28:25.390124 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.390098    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c647346d-70fe-456e-b81e-f3e90f7bbf17-istiod-ca-cert\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz\" (UID: \"c647346d-70fe-456e-b81e-f3e90f7bbf17\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"
Oct 30 22:28:25.391652 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.391615    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c647346d-70fe-456e-b81e-f3e90f7bbf17-istio-envoy\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz\" (UID: \"c647346d-70fe-456e-b81e-f3e90f7bbf17\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"
Oct 30 22:28:25.391875 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.391855    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c647346d-70fe-456e-b81e-f3e90f7bbf17-istio-podinfo\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz\" (UID: \"c647346d-70fe-456e-b81e-f3e90f7bbf17\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"
Oct 30 22:28:25.453676 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.453631    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c647346d-70fe-456e-b81e-f3e90f7bbf17-istio-token\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz\" (UID: \"c647346d-70fe-456e-b81e-f3e90f7bbf17\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"
Oct 30 22:28:25.459044 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.459011    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lk6d\" (UniqueName: \"kubernetes.io/projected/c647346d-70fe-456e-b81e-f3e90f7bbf17-kube-api-access-6lk6d\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz\" (UID: \"c647346d-70fe-456e-b81e-f3e90f7bbf17\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"
Oct 30 22:28:25.550835 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.550730    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"
Oct 30 22:28:25.988016 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:28:25.987969    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc647346d_70fe_456e_b81e_f3e90f7bbf17.slice/crio-053e88db4d5e894336d879ffe93a8658fd0a60b286a32e8add60f30ffd1eeff2 WatchSource:0}: Error finding container 053e88db4d5e894336d879ffe93a8658fd0a60b286a32e8add60f30ffd1eeff2: Status 404 returned error can't find the container with id 053e88db4d5e894336d879ffe93a8658fd0a60b286a32e8add60f30ffd1eeff2
Oct 30 22:28:25.990714 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.990673    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:28:25.990823 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.990748    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:28:25.990823 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.990787    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:28:25.991937 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:25.991912    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"]
Oct 30 22:28:26.486409 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:26.486372    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz" event={"ID":"c647346d-70fe-456e-b81e-f3e90f7bbf17","Type":"ContainerStarted","Data":"cc5d31fdc84fef9edc42a0f74c93fa929a60a721019709924f398147d30c6acd"}
Oct 30 22:28:26.486837 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:26.486428    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz" event={"ID":"c647346d-70fe-456e-b81e-f3e90f7bbf17","Type":"ContainerStarted","Data":"053e88db4d5e894336d879ffe93a8658fd0a60b286a32e8add60f30ffd1eeff2"}
Oct 30 22:28:26.551127 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:26.551088    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"
Oct 30 22:28:26.614423 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:26.614353    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz" podStartSLOduration=1.6143313099999999 podStartE2EDuration="1.61433131s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:28:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:28:26.611520149 +0000 UTC m=+3048.789008766" watchObservedRunningTime="2026-10-30 22:28:26.61433131 +0000 UTC m=+3048.791819928"
Oct 30 22:28:27.553472 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:27.553421    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused" start-of-body=
Oct 30 22:28:27.553984 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:27.553495    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz" podUID="c647346d-70fe-456e-b81e-f3e90f7bbf17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused"
Oct 30 22:28:28.256125 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:28.256048    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:28.552013 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:28.551919    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused" start-of-body=
Oct 30 22:28:28.552013 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:28.551991    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz" podUID="c647346d-70fe-456e-b81e-f3e90f7bbf17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused"
Oct 30 22:28:29.552128 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:29.552086    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused" start-of-body=
Oct 30 22:28:29.552608 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:29.552157    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz" podUID="c647346d-70fe-456e-b81e-f3e90f7bbf17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused"
Oct 30 22:28:30.551736 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:30.551691    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused" start-of-body=
Oct 30 22:28:30.551947 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:30.551754    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz" podUID="c647346d-70fe-456e-b81e-f3e90f7bbf17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused"
Oct 30 22:28:31.551607 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:31.551572    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused" start-of-body=
Oct 30 22:28:31.552036 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:31.551622    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz" podUID="c647346d-70fe-456e-b81e-f3e90f7bbf17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused"
Oct 30 22:28:32.552025 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:32.551985    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused" start-of-body=
Oct 30 22:28:32.552571 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:32.552070    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz" podUID="c647346d-70fe-456e-b81e-f3e90f7bbf17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused"
Oct 30 22:28:32.930751 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:32.930709    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:33.970670 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:33.969879    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"
Oct 30 22:28:33.972199 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:33.972173    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"
Oct 30 22:28:33.990997 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:33.990955    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"
Oct 30 22:28:35.783263 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:28:35.783037    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:28:37.827583 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:37.827552    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:42.421750 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:42.421712    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:45.907722 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:45.907690    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:46.277155 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:46.277077    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:46.442835 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:46.442802    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:46.507083 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:46.507045    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"]
Oct 30 22:28:46.511028 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:46.510987    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"
Oct 30 22:28:46.567196 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:46.567095    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-5e884c-openshift-default-dockercfg-s2jjh\""
Oct 30 22:28:46.675492 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:46.671698    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zd8vg\" (UniqueName: \"kubernetes.io/projected/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-kube-api-access-zd8vg\") pod \"e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x\" (UID: \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\") " pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"
Oct 30 22:28:46.675844 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:46.675810    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-istio-token\") pod \"e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x\" (UID: \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\") " pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"
Oct 30 22:28:46.676076 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:46.676057    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-istio-podinfo\") pod \"e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x\" (UID: \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\") " pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"
Oct 30 22:28:46.676214 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:46.676196    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-credential-socket\") pod \"e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x\" (UID: \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\") " pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"
Oct 30 22:28:46.676382 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:46.676364    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-istio-envoy\") pod \"e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x\" (UID: \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\") " pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"
Oct 30 22:28:46.680470 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:46.680442    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-istio-data\") pod \"e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x\" (UID: \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\") " pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"
Oct 30 22:28:46.680763 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:46.680738    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-workload-socket\") pod \"e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x\" (UID: \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\") " pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"
Oct 30 22:28:46.680996 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:46.680976    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-istiod-ca-cert\") pod \"e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x\" (UID: \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\") " pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"
Oct 30 22:28:46.681187 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:46.681167    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-workload-certs\") pod \"e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x\" (UID: \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\") " pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"
Oct 30 22:28:46.681431 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:46.681188    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:46.685320 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:46.685293    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"]
Oct 30 22:28:46.782742 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:46.782697    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-istio-podinfo\") pod \"e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x\" (UID: \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\") " pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"
Oct 30 22:28:46.782961 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:46.782750    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-credential-socket\") pod \"e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x\" (UID: \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\") " pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"
Oct 30 22:28:46.782961 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:46.782791    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-istio-envoy\") pod \"e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x\" (UID: \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\") " pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"
Oct 30 22:28:46.782961 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:46.782821    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-istio-data\") pod \"e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x\" (UID: \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\") " pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"
Oct 30 22:28:46.782961 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:46.782886    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-workload-socket\") pod \"e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x\" (UID: \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\") " pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"
Oct 30 22:28:46.783204 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:46.782975    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-istiod-ca-cert\") pod \"e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x\" (UID: \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\") " pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"
Oct 30 22:28:46.783204 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:46.783105    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-workload-certs\") pod \"e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x\" (UID: \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\") " pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"
Oct 30 22:28:46.783204 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:46.783169    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zd8vg\" (UniqueName: \"kubernetes.io/projected/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-kube-api-access-zd8vg\") pod \"e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x\" (UID: \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\") " pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"
Oct 30 22:28:46.783204 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:46.783179    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-credential-socket\") pod \"e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x\" (UID: \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\") " pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"
Oct 30 22:28:46.783465 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:46.783215    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-istio-token\") pod \"e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x\" (UID: \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\") " pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"
Oct 30 22:28:46.783465 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:46.783309    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-istio-data\") pod \"e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x\" (UID: \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\") " pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"
Oct 30 22:28:46.783702 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:46.783681    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-workload-socket\") pod \"e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x\" (UID: \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\") " pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"
Oct 30 22:28:46.783813 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:46.783796    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-workload-certs\") pod \"e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x\" (UID: \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\") " pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"
Oct 30 22:28:46.784002 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:46.783980    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-istiod-ca-cert\") pod \"e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x\" (UID: \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\") " pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"
Oct 30 22:28:46.785492 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:46.785469    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-istio-envoy\") pod \"e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x\" (UID: \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\") " pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"
Oct 30 22:28:46.785683 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:46.785666    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-istio-podinfo\") pod \"e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x\" (UID: \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\") " pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"
Oct 30 22:28:46.872353 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:46.871802    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:46.879221 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:46.873368    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-istio-token\") pod \"e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x\" (UID: \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\") " pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"
Oct 30 22:28:46.879221 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:46.873573    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zd8vg\" (UniqueName: \"kubernetes.io/projected/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-kube-api-access-zd8vg\") pod \"e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x\" (UID: \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\") " pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"
Oct 30 22:28:47.102554 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:47.102515    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:47.140314 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:47.135509    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"
Oct 30 22:28:47.590312 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:28:47.590277    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef5ef1c4_ec4f_4336_b2d4_9c727dc58bea.slice/crio-e2328916e36c247a3268e19d96012f1f17893717728fa6e2c6d5d295f90211f4 WatchSource:0}: Error finding container e2328916e36c247a3268e19d96012f1f17893717728fa6e2c6d5d295f90211f4: Status 404 returned error can't find the container with id e2328916e36c247a3268e19d96012f1f17893717728fa6e2c6d5d295f90211f4
Oct 30 22:28:47.593295 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:47.593229    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:28:47.593446 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:47.593336    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:28:47.593446 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:47.593381    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:28:47.594944 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:47.594915    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"]
Oct 30 22:28:47.600618 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:47.600588    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:47.725323 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:47.725288    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x" event={"ID":"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea","Type":"ContainerStarted","Data":"b5e0536939bf1528ad23a40a01e4190d2a8a8b88d444591654dd83060b68595a"}
Oct 30 22:28:47.725860 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:47.725349    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x" event={"ID":"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea","Type":"ContainerStarted","Data":"e2328916e36c247a3268e19d96012f1f17893717728fa6e2c6d5d295f90211f4"}
Oct 30 22:28:47.853861 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:47.853735    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x" podStartSLOduration=1.853714139 podStartE2EDuration="1.853714139s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:28:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:28:47.852868948 +0000 UTC m=+3070.030357566" watchObservedRunningTime="2026-10-30 22:28:47.853714139 +0000 UTC m=+3070.031202758"
Oct 30 22:28:48.136022 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:48.135988    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"
Oct 30 22:28:48.141822 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:48.141797    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"
Oct 30 22:28:48.544720 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:48.544638    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:48.732367 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:48.732330    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"
Oct 30 22:28:48.733308 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:48.733285    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"
Oct 30 22:28:49.087172 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:49.087135    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"]
Oct 30 22:28:49.091228 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:49.091208    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"
Oct 30 22:28:49.141259 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:49.123099    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-credential-socket\") pod \"e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s\" (UID: \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\") " pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"
Oct 30 22:28:49.142018 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:49.141987    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvztn\" (UniqueName: \"kubernetes.io/projected/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-kube-api-access-jvztn\") pod \"e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s\" (UID: \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\") " pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"
Oct 30 22:28:49.142357 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:49.142332    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-workload-certs\") pod \"e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s\" (UID: \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\") " pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"
Oct 30 22:28:49.142648 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:49.142617    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-istiod-ca-cert\") pod \"e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s\" (UID: \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\") " pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"
Oct 30 22:28:49.142849 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:49.142834    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-workload-socket\") pod \"e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s\" (UID: \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\") " pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"
Oct 30 22:28:49.143006 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:49.142990    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-istio-data\") pod \"e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s\" (UID: \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\") " pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"
Oct 30 22:28:49.143163 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:49.143148    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-istio-token\") pod \"e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s\" (UID: \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\") " pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"
Oct 30 22:28:49.143380 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:49.143364    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-istio-podinfo\") pod \"e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s\" (UID: \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\") " pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"
Oct 30 22:28:49.143595 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:49.143580    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-istio-envoy\") pod \"e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s\" (UID: \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\") " pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"
Oct 30 22:28:49.156490 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:49.156457    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-43d592-openshift-default-dockercfg-ntzbs\""
Oct 30 22:28:49.200501 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:49.193022    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:49.214433 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:49.209616    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"]
Oct 30 22:28:49.245446 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:49.245391    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-istiod-ca-cert\") pod \"e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s\" (UID: \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\") " pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"
Oct 30 22:28:49.245630 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:49.245494    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-workload-socket\") pod \"e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s\" (UID: \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\") " pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"
Oct 30 22:28:49.245630 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:49.245521    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-istio-data\") pod \"e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s\" (UID: \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\") " pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"
Oct 30 22:28:49.245630 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:49.245558    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-istio-token\") pod \"e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s\" (UID: \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\") " pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"
Oct 30 22:28:49.245822 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:49.245634    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-istio-podinfo\") pod \"e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s\" (UID: \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\") " pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"
Oct 30 22:28:49.245822 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:49.245717    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-istio-envoy\") pod \"e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s\" (UID: \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\") " pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"
Oct 30 22:28:49.245822 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:49.245765    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-credential-socket\") pod \"e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s\" (UID: \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\") " pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"
Oct 30 22:28:49.245822 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:49.245812    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvztn\" (UniqueName: \"kubernetes.io/projected/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-kube-api-access-jvztn\") pod \"e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s\" (UID: \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\") " pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"
Oct 30 22:28:49.246036 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:49.245858    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-workload-certs\") pod \"e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s\" (UID: \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\") " pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"
Oct 30 22:28:49.246036 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:49.245964    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-istio-data\") pod \"e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s\" (UID: \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\") " pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"
Oct 30 22:28:49.246158 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:49.246092    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-workload-socket\") pod \"e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s\" (UID: \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\") " pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"
Oct 30 22:28:49.248280 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:49.246280    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-workload-certs\") pod \"e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s\" (UID: \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\") " pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"
Oct 30 22:28:49.248280 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:49.246306    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-credential-socket\") pod \"e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s\" (UID: \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\") " pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"
Oct 30 22:28:49.248280 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:49.246324    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-istiod-ca-cert\") pod \"e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s\" (UID: \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\") " pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"
Oct 30 22:28:49.248946 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:49.248920    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-istio-envoy\") pod \"e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s\" (UID: \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\") " pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"
Oct 30 22:28:49.249294 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:49.249220    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-istio-podinfo\") pod \"e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s\" (UID: \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\") " pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"
Oct 30 22:28:49.310893 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:49.310860    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvztn\" (UniqueName: \"kubernetes.io/projected/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-kube-api-access-jvztn\") pod \"e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s\" (UID: \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\") " pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"
Oct 30 22:28:49.314997 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:49.314959    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-istio-token\") pod \"e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s\" (UID: \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\") " pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"
Oct 30 22:28:49.405796 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:49.405759    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"
Oct 30 22:28:49.575102 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:49.575066    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:49.842974 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:28:49.842941    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod239ef6a2_b9f8_4247_9db4_fc4192f6bf1c.slice/crio-690386ef1dc45876f8d3a38792bf6fe64cc7b717e085082bdf037fd8c980a301 WatchSource:0}: Error finding container 690386ef1dc45876f8d3a38792bf6fe64cc7b717e085082bdf037fd8c980a301: Status 404 returned error can't find the container with id 690386ef1dc45876f8d3a38792bf6fe64cc7b717e085082bdf037fd8c980a301
Oct 30 22:28:49.845874 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:49.845836    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:28:49.845998 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:49.845925    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:28:49.845998 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:49.845966    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:28:49.847767 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:49.847743    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"]
Oct 30 22:28:50.143003 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.142957    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"]
Oct 30 22:28:50.147793 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.147760    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"
Oct 30 22:28:50.170788 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.170756    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"]
Oct 30 22:28:50.177794 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.177747    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"
Oct 30 22:28:50.205967 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.205941    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-56f3fe-openshift-default-dockercfg-48wbx\""
Oct 30 22:28:50.226142 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.226106    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"]
Oct 30 22:28:50.263629 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.261574    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"]
Oct 30 22:28:50.263629 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.262366    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-workload-certs\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb\" (UID: \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"
Oct 30 22:28:50.263629 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.262414    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-istio-data\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6\" (UID: \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"
Oct 30 22:28:50.263629 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.262458    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-istio-podinfo\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb\" (UID: \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"
Oct 30 22:28:50.263629 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.262505    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-workload-certs\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6\" (UID: \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"
Oct 30 22:28:50.263629 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.262555    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brdl4\" (UniqueName: \"kubernetes.io/projected/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-kube-api-access-brdl4\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6\" (UID: \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"
Oct 30 22:28:50.263629 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.262586    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-workload-socket\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6\" (UID: \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"
Oct 30 22:28:50.263629 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.262613    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-istio-token\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb\" (UID: \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"
Oct 30 22:28:50.263629 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.262684    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-istiod-ca-cert\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6\" (UID: \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"
Oct 30 22:28:50.263629 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.262717    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-istio-token\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6\" (UID: \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"
Oct 30 22:28:50.263629 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.262747    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-workload-socket\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb\" (UID: \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"
Oct 30 22:28:50.263629 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.262773    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-istio-podinfo\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6\" (UID: \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"
Oct 30 22:28:50.263629 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.262825    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-istio-data\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb\" (UID: \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"
Oct 30 22:28:50.263629 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.262866    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-credential-socket\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb\" (UID: \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"
Oct 30 22:28:50.263629 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.262922    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6j2f\" (UniqueName: \"kubernetes.io/projected/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-kube-api-access-x6j2f\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb\" (UID: \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"
Oct 30 22:28:50.263629 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.262950    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-istio-envoy\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb\" (UID: \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"
Oct 30 22:28:50.263629 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.262991    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-istiod-ca-cert\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb\" (UID: \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"
Oct 30 22:28:50.263629 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.263049    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-credential-socket\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6\" (UID: \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"
Oct 30 22:28:50.263629 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.263073    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-istio-envoy\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6\" (UID: \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"
Oct 30 22:28:50.364888 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.364851    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-istiod-ca-cert\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb\" (UID: \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"
Oct 30 22:28:50.365107 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.364927    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-credential-socket\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6\" (UID: \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"
Oct 30 22:28:50.365107 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.364955    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-istio-envoy\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6\" (UID: \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"
Oct 30 22:28:50.365107 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.364996    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-workload-certs\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb\" (UID: \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"
Oct 30 22:28:50.365107 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.365027    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-istio-data\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6\" (UID: \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"
Oct 30 22:28:50.365107 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.365068    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-istio-podinfo\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb\" (UID: \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"
Oct 30 22:28:50.365107 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.365096    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-workload-certs\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6\" (UID: \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"
Oct 30 22:28:50.365543 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.365147    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brdl4\" (UniqueName: \"kubernetes.io/projected/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-kube-api-access-brdl4\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6\" (UID: \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"
Oct 30 22:28:50.365543 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.365179    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-workload-socket\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6\" (UID: \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"
Oct 30 22:28:50.365543 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.365206    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-istio-token\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb\" (UID: \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"
Oct 30 22:28:50.365543 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.365289    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-istiod-ca-cert\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6\" (UID: \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"
Oct 30 22:28:50.365543 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.365324    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-istio-token\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6\" (UID: \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"
Oct 30 22:28:50.365543 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.365356    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-workload-socket\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb\" (UID: \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"
Oct 30 22:28:50.365543 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.365389    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-istio-podinfo\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6\" (UID: \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"
Oct 30 22:28:50.365543 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.365427    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-credential-socket\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6\" (UID: \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"
Oct 30 22:28:50.365543 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.365471    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-istio-data\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb\" (UID: \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"
Oct 30 22:28:50.365543 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.365493    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-workload-certs\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb\" (UID: \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"
Oct 30 22:28:50.365543 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.365520    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-credential-socket\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb\" (UID: \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"
Oct 30 22:28:50.365543 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.365545    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-istio-data\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6\" (UID: \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"
Oct 30 22:28:50.366771 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.365607    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6j2f\" (UniqueName: \"kubernetes.io/projected/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-kube-api-access-x6j2f\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb\" (UID: \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"
Oct 30 22:28:50.366771 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.365648    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-istio-envoy\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb\" (UID: \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"
Oct 30 22:28:50.366771 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.365724    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-workload-socket\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6\" (UID: \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"
Oct 30 22:28:50.366771 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.365795    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-istiod-ca-cert\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb\" (UID: \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"
Oct 30 22:28:50.366771 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.365863    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-credential-socket\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb\" (UID: \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"
Oct 30 22:28:50.366771 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.365913    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-workload-certs\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6\" (UID: \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"
Oct 30 22:28:50.366771 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.366103    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-workload-socket\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb\" (UID: \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"
Oct 30 22:28:50.366771 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.366148    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-istio-data\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb\" (UID: \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"
Oct 30 22:28:50.366771 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.366535    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-istiod-ca-cert\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6\" (UID: \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"
Oct 30 22:28:50.368155 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.368126    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-istio-envoy\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb\" (UID: \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"
Oct 30 22:28:50.368321 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.368167    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-istio-envoy\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6\" (UID: \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"
Oct 30 22:28:50.368392 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.368340    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-istio-podinfo\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6\" (UID: \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"
Oct 30 22:28:50.368516 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.368497    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-istio-podinfo\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb\" (UID: \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"
Oct 30 22:28:50.433075 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.432991    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-istio-token\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6\" (UID: \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"
Oct 30 22:28:50.433075 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.433040    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6j2f\" (UniqueName: \"kubernetes.io/projected/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-kube-api-access-x6j2f\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb\" (UID: \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"
Oct 30 22:28:50.433450 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.433114    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-istio-token\") pod \"e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb\" (UID: \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\") " pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"
Oct 30 22:28:50.433450 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.433267    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-brdl4\" (UniqueName: \"kubernetes.io/projected/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-kube-api-access-brdl4\") pod \"e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6\" (UID: \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\") " pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"
Oct 30 22:28:50.462487 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.462455    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"
Oct 30 22:28:50.502876 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.502831    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"
Oct 30 22:28:50.597358 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.597277    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:50.754269 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.754164    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s" event={"ID":"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c","Type":"ContainerStarted","Data":"373bdb1823397318e10f3e052140772b7c13846f314a3bb9840627af7ca2c82a"}
Oct 30 22:28:50.754453 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.754271    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s" event={"ID":"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c","Type":"ContainerStarted","Data":"690386ef1dc45876f8d3a38792bf6fe64cc7b717e085082bdf037fd8c980a301"}
Oct 30 22:28:50.767446 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.767417    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:50.882827 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.882776    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s" podStartSLOduration=1.882761186 podStartE2EDuration="1.882761186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:28:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:28:50.881270621 +0000 UTC m=+3073.058759238" watchObservedRunningTime="2026-10-30 22:28:50.882761186 +0000 UTC m=+3073.060249801"
Oct 30 22:28:50.927755 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:28:50.927714    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode631c7a2_bd8e_4a9f_8768_6fa43cde6080.slice/crio-9442901bbb9e2ef0a764802aae175d2bb32ddc239ecd566e3a2e369b0875c5cb WatchSource:0}: Error finding container 9442901bbb9e2ef0a764802aae175d2bb32ddc239ecd566e3a2e369b0875c5cb: Status 404 returned error can't find the container with id 9442901bbb9e2ef0a764802aae175d2bb32ddc239ecd566e3a2e369b0875c5cb
Oct 30 22:28:50.930297 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.930218    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:28:50.930407 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.930365    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:28:50.930407 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.930399    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:28:50.937229 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.937203    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"]
Oct 30 22:28:50.947737 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:28:50.947704    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15fb5463_5e07_43b5_bfb9_f7b1b635c8d6.slice/crio-8d013afa14574e53b7de751de20015476651bb818f182e55a39600d6a8a30cf6 WatchSource:0}: Error finding container 8d013afa14574e53b7de751de20015476651bb818f182e55a39600d6a8a30cf6: Status 404 returned error can't find the container with id 8d013afa14574e53b7de751de20015476651bb818f182e55a39600d6a8a30cf6
Oct 30 22:28:50.948005 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.947975    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:50.950347 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.950314    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:28:50.950459 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.950383    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:28:50.950459 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.950415    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:28:50.952422 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:50.952396    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"]
Oct 30 22:28:51.406793 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:51.406755    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"
Oct 30 22:28:51.413503 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:51.413469    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"
Oct 30 22:28:51.765459 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:51.764905    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6" event={"ID":"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6","Type":"ContainerStarted","Data":"08384472cf5c0b765e58e2e2ed25d17634880c583767b0b21c41f3669ed6ebf5"}
Oct 30 22:28:51.765459 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:51.764972    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6" event={"ID":"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6","Type":"ContainerStarted","Data":"8d013afa14574e53b7de751de20015476651bb818f182e55a39600d6a8a30cf6"}
Oct 30 22:28:51.767171 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:51.767143    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb" event={"ID":"e631c7a2-bd8e-4a9f-8768-6fa43cde6080","Type":"ContainerStarted","Data":"74b8bf64a7107c96a9243e2797b6593aa4c62bef354a2db709b66dedbb741d00"}
Oct 30 22:28:51.767331 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:51.767195    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb" event={"ID":"e631c7a2-bd8e-4a9f-8768-6fa43cde6080","Type":"ContainerStarted","Data":"9442901bbb9e2ef0a764802aae175d2bb32ddc239ecd566e3a2e369b0875c5cb"}
Oct 30 22:28:51.767574 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:51.767551    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"
Oct 30 22:28:51.768387 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:51.768369    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"
Oct 30 22:28:51.899726 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:51.899673    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6" podStartSLOduration=1.89965355 podStartE2EDuration="1.89965355s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:28:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:28:51.897020724 +0000 UTC m=+3074.074509334" watchObservedRunningTime="2026-10-30 22:28:51.89965355 +0000 UTC m=+3074.077142166"
Oct 30 22:28:52.034205 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:52.034074    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb" podStartSLOduration=2.034052072 podStartE2EDuration="2.034052072s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:28:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:28:52.030731033 +0000 UTC m=+3074.208219650" watchObservedRunningTime="2026-10-30 22:28:52.034052072 +0000 UTC m=+3074.211540691"
Oct 30 22:28:52.081025 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:52.080992    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:52.464644 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:52.464304    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"
Oct 30 22:28:52.465043 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:52.464703    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused" start-of-body=
Oct 30 22:28:52.465043 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:52.464763    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb" podUID="e631c7a2-bd8e-4a9f-8768-6fa43cde6080" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused"
Oct 30 22:28:52.513273 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:52.513020    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"
Oct 30 22:28:52.513273 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:52.513158    4871 patch_prober.go:29] interesting pod/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.175:15021/healthz/ready\": dial tcp 10.133.0.175:15021: connect: connection refused" start-of-body=
Oct 30 22:28:52.513273 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:52.513203    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6" podUID="15fb5463-5e07-43b5-bfb9-f7b1b635c8d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.175:15021/healthz/ready\": dial tcp 10.133.0.175:15021: connect: connection refused"
Oct 30 22:28:53.463264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:53.462953    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused" start-of-body=
Oct 30 22:28:53.463264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:53.463017    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb" podUID="e631c7a2-bd8e-4a9f-8768-6fa43cde6080" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused"
Oct 30 22:28:53.506694 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:53.506654    4871 patch_prober.go:29] interesting pod/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.175:15021/healthz/ready\": dial tcp 10.133.0.175:15021: connect: connection refused" start-of-body=
Oct 30 22:28:53.507311 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:53.507276    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6" podUID="15fb5463-5e07-43b5-bfb9-f7b1b635c8d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.175:15021/healthz/ready\": dial tcp 10.133.0.175:15021: connect: connection refused"
Oct 30 22:28:53.956262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:53.955132    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:54.153262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:54.152768    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:54.468408 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:54.468368    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused" start-of-body=
Oct 30 22:28:54.468697 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:54.468665    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb" podUID="e631c7a2-bd8e-4a9f-8768-6fa43cde6080" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused"
Oct 30 22:28:54.508700 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:54.508661    4871 patch_prober.go:29] interesting pod/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.175:15021/healthz/ready\": dial tcp 10.133.0.175:15021: connect: connection refused" start-of-body=
Oct 30 22:28:54.509295 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:54.509257    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6" podUID="15fb5463-5e07-43b5-bfb9-f7b1b635c8d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.175:15021/healthz/ready\": dial tcp 10.133.0.175:15021: connect: connection refused"
Oct 30 22:28:55.463552 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:55.463506    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused" start-of-body=
Oct 30 22:28:55.463728 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:55.463573    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb" podUID="e631c7a2-bd8e-4a9f-8768-6fa43cde6080" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused"
Oct 30 22:28:55.510266 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:55.508656    4871 patch_prober.go:29] interesting pod/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.175:15021/healthz/ready\": dial tcp 10.133.0.175:15021: connect: connection refused" start-of-body=
Oct 30 22:28:55.510266 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:55.508721    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6" podUID="15fb5463-5e07-43b5-bfb9-f7b1b635c8d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.175:15021/healthz/ready\": dial tcp 10.133.0.175:15021: connect: connection refused"
Oct 30 22:28:56.473262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:56.472853    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused" start-of-body=
Oct 30 22:28:56.473262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:56.472928    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb" podUID="e631c7a2-bd8e-4a9f-8768-6fa43cde6080" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused"
Oct 30 22:28:56.555412 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:56.555367    4871 patch_prober.go:29] interesting pod/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.175:15021/healthz/ready\": dial tcp 10.133.0.175:15021: connect: connection refused" start-of-body=
Oct 30 22:28:56.565432 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:56.565356    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6" podUID="15fb5463-5e07-43b5-bfb9-f7b1b635c8d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.175:15021/healthz/ready\": dial tcp 10.133.0.175:15021: connect: connection refused"
Oct 30 22:28:57.463112 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:57.463070    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused" start-of-body=
Oct 30 22:28:57.463310 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:57.463137    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb" podUID="e631c7a2-bd8e-4a9f-8768-6fa43cde6080" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused"
Oct 30 22:28:57.504287 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:57.504221    4871 patch_prober.go:29] interesting pod/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.175:15021/healthz/ready\": dial tcp 10.133.0.175:15021: connect: connection refused" start-of-body=
Oct 30 22:28:57.504548 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:57.504520    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6" podUID="15fb5463-5e07-43b5-bfb9-f7b1b635c8d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.175:15021/healthz/ready\": dial tcp 10.133.0.175:15021: connect: connection refused"
Oct 30 22:28:57.608907 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:57.608871    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:58.750847 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:58.750813    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"
Oct 30 22:28:58.751656 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:58.751635    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"
Oct 30 22:28:58.752755 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:58.752733    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"
Oct 30 22:28:58.817348 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:58.817302    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"
Oct 30 22:28:58.817721 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:58.817696    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"
Oct 30 22:28:58.818917 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:28:58.818892    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"
Oct 30 22:29:01.583102 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:01.583068    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:03.278381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:03.278290    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:03.620263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:03.618558    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:03.850731 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:03.850695    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:04.268357 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:04.268314    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:04.432263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:04.428867    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:04.608394 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:04.608294    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:05.023882 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:05.023738    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.171:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:29:05.023882 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:05.023809    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz" podUID="c647346d-70fe-456e-b81e-f3e90f7bbf17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.171:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:29:05.763271 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:29:05.763201    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:29:06.126574 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:06.126541    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:06.667754 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:06.667714    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"]
Oct 30 22:29:06.673206 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:06.673186    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"
Oct 30 22:29:06.741264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:06.738450    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w1-ef00a4-openshift-default-dockercfg-c8bxv\""
Oct 30 22:29:06.864106 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:06.864051    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cbecab8-bfad-40bf-850a-905e4d365d29-istio-data\") pod \"e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp\" (UID: \"6cbecab8-bfad-40bf-850a-905e4d365d29\") " pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"
Oct 30 22:29:06.865014 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:06.864986    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cbecab8-bfad-40bf-850a-905e4d365d29-workload-socket\") pod \"e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp\" (UID: \"6cbecab8-bfad-40bf-850a-905e4d365d29\") " pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"
Oct 30 22:29:06.865257 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:06.865219    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cbecab8-bfad-40bf-850a-905e4d365d29-istio-podinfo\") pod \"e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp\" (UID: \"6cbecab8-bfad-40bf-850a-905e4d365d29\") " pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"
Oct 30 22:29:06.865652 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:06.865614    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnq9r\" (UniqueName: \"kubernetes.io/projected/6cbecab8-bfad-40bf-850a-905e4d365d29-kube-api-access-xnq9r\") pod \"e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp\" (UID: \"6cbecab8-bfad-40bf-850a-905e4d365d29\") " pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"
Oct 30 22:29:06.865851 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:06.865834    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cbecab8-bfad-40bf-850a-905e4d365d29-istiod-ca-cert\") pod \"e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp\" (UID: \"6cbecab8-bfad-40bf-850a-905e4d365d29\") " pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"
Oct 30 22:29:06.866000 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:06.865985    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6cbecab8-bfad-40bf-850a-905e4d365d29-istio-token\") pod \"e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp\" (UID: \"6cbecab8-bfad-40bf-850a-905e4d365d29\") " pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"
Oct 30 22:29:06.866213 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:06.866184    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cbecab8-bfad-40bf-850a-905e4d365d29-workload-certs\") pod \"e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp\" (UID: \"6cbecab8-bfad-40bf-850a-905e4d365d29\") " pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"
Oct 30 22:29:06.866393 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:06.866375    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cbecab8-bfad-40bf-850a-905e4d365d29-credential-socket\") pod \"e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp\" (UID: \"6cbecab8-bfad-40bf-850a-905e4d365d29\") " pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"
Oct 30 22:29:06.866598 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:06.866548    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cbecab8-bfad-40bf-850a-905e4d365d29-istio-envoy\") pod \"e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp\" (UID: \"6cbecab8-bfad-40bf-850a-905e4d365d29\") " pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"
Oct 30 22:29:06.944183 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:06.922156    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:06.944183 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:06.924436    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"]
Oct 30 22:29:06.968286 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:06.968194    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cbecab8-bfad-40bf-850a-905e4d365d29-istiod-ca-cert\") pod \"e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp\" (UID: \"6cbecab8-bfad-40bf-850a-905e4d365d29\") " pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"
Oct 30 22:29:06.968286 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:06.968261    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6cbecab8-bfad-40bf-850a-905e4d365d29-istio-token\") pod \"e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp\" (UID: \"6cbecab8-bfad-40bf-850a-905e4d365d29\") " pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"
Oct 30 22:29:06.968555 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:06.968314    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cbecab8-bfad-40bf-850a-905e4d365d29-workload-certs\") pod \"e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp\" (UID: \"6cbecab8-bfad-40bf-850a-905e4d365d29\") " pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"
Oct 30 22:29:06.968555 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:06.968345    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cbecab8-bfad-40bf-850a-905e4d365d29-credential-socket\") pod \"e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp\" (UID: \"6cbecab8-bfad-40bf-850a-905e4d365d29\") " pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"
Oct 30 22:29:06.968555 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:06.968385    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cbecab8-bfad-40bf-850a-905e4d365d29-istio-envoy\") pod \"e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp\" (UID: \"6cbecab8-bfad-40bf-850a-905e4d365d29\") " pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"
Oct 30 22:29:06.968555 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:06.968433    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cbecab8-bfad-40bf-850a-905e4d365d29-istio-data\") pod \"e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp\" (UID: \"6cbecab8-bfad-40bf-850a-905e4d365d29\") " pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"
Oct 30 22:29:06.968555 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:06.968479    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cbecab8-bfad-40bf-850a-905e4d365d29-workload-socket\") pod \"e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp\" (UID: \"6cbecab8-bfad-40bf-850a-905e4d365d29\") " pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"
Oct 30 22:29:06.968555 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:06.968537    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cbecab8-bfad-40bf-850a-905e4d365d29-istio-podinfo\") pod \"e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp\" (UID: \"6cbecab8-bfad-40bf-850a-905e4d365d29\") " pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"
Oct 30 22:29:06.968944 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:06.968700    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnq9r\" (UniqueName: \"kubernetes.io/projected/6cbecab8-bfad-40bf-850a-905e4d365d29-kube-api-access-xnq9r\") pod \"e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp\" (UID: \"6cbecab8-bfad-40bf-850a-905e4d365d29\") " pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"
Oct 30 22:29:06.973575 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:06.969950    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cbecab8-bfad-40bf-850a-905e4d365d29-istiod-ca-cert\") pod \"e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp\" (UID: \"6cbecab8-bfad-40bf-850a-905e4d365d29\") " pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"
Oct 30 22:29:06.973575 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:06.970605    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cbecab8-bfad-40bf-850a-905e4d365d29-workload-certs\") pod \"e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp\" (UID: \"6cbecab8-bfad-40bf-850a-905e4d365d29\") " pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"
Oct 30 22:29:06.973575 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:06.970841    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cbecab8-bfad-40bf-850a-905e4d365d29-credential-socket\") pod \"e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp\" (UID: \"6cbecab8-bfad-40bf-850a-905e4d365d29\") " pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"
Oct 30 22:29:06.981080 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:06.981042    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cbecab8-bfad-40bf-850a-905e4d365d29-workload-socket\") pod \"e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp\" (UID: \"6cbecab8-bfad-40bf-850a-905e4d365d29\") " pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"
Oct 30 22:29:06.987290 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:06.987247    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cbecab8-bfad-40bf-850a-905e4d365d29-istio-data\") pod \"e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp\" (UID: \"6cbecab8-bfad-40bf-850a-905e4d365d29\") " pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"
Oct 30 22:29:06.990513 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:06.990482    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cbecab8-bfad-40bf-850a-905e4d365d29-istio-podinfo\") pod \"e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp\" (UID: \"6cbecab8-bfad-40bf-850a-905e4d365d29\") " pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"
Oct 30 22:29:06.995685 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:06.995639    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cbecab8-bfad-40bf-850a-905e4d365d29-istio-envoy\") pod \"e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp\" (UID: \"6cbecab8-bfad-40bf-850a-905e4d365d29\") " pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"
Oct 30 22:29:07.038361 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:07.038325    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnq9r\" (UniqueName: \"kubernetes.io/projected/6cbecab8-bfad-40bf-850a-905e4d365d29-kube-api-access-xnq9r\") pod \"e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp\" (UID: \"6cbecab8-bfad-40bf-850a-905e4d365d29\") " pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"
Oct 30 22:29:07.038563 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:07.038532    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6cbecab8-bfad-40bf-850a-905e4d365d29-istio-token\") pod \"e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp\" (UID: \"6cbecab8-bfad-40bf-850a-905e4d365d29\") " pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"
Oct 30 22:29:07.149596 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:07.149552    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:07.288048 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:07.287964    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"
Oct 30 22:29:07.705226 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:07.705192    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:07.733340 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:29:07.733299    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cbecab8_bfad_40bf_850a_905e4d365d29.slice/crio-bfb38a5492650097a945fd954d80ea1f607d3456127b840e837c5534d0e5041b WatchSource:0}: Error finding container bfb38a5492650097a945fd954d80ea1f607d3456127b840e837c5534d0e5041b: Status 404 returned error can't find the container with id bfb38a5492650097a945fd954d80ea1f607d3456127b840e837c5534d0e5041b
Oct 30 22:29:07.737446 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:07.737408    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:29:07.737570 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:07.737500    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:29:07.737570 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:07.737538    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:29:07.740040 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:07.739909    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"]
Oct 30 22:29:07.911059 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:07.911022    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:08.023329 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.021584    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp" event={"ID":"6cbecab8-bfad-40bf-850a-905e4d365d29","Type":"ContainerStarted","Data":"a2cfedda456012711321024d33b23567686adf80d7cc9f2cbc8f9f485fbc4d08"}
Oct 30 22:29:08.023329 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.022374    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp" event={"ID":"6cbecab8-bfad-40bf-850a-905e4d365d29","Type":"ContainerStarted","Data":"bfb38a5492650097a945fd954d80ea1f607d3456127b840e837c5534d0e5041b"}
Oct 30 22:29:08.144274 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.139614    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-43d592-54c9b96dcf-6bwjj"]
Oct 30 22:29:08.151570 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.151535    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-43d592-54c9b96dcf-6bwjj"
Oct 30 22:29:08.158638 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.158574    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp" podStartSLOduration=2.158553912 podStartE2EDuration="2.158553912s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:29:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:29:08.157002933 +0000 UTC m=+3090.334491549" watchObservedRunningTime="2026-10-30 22:29:08.158553912 +0000 UTC m=+3090.336042530"
Oct 30 22:29:08.191218 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.191182    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:08.195262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.195218    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-43d592-54c9b96dcf-6bwjj"]
Oct 30 22:29:08.208267 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.204286    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8gzw\" (UniqueName: \"kubernetes.io/projected/0c63b14b-1649-4c30-b788-81e4e75e40f4-kube-api-access-h8gzw\") pod \"payload-pre-processing-e2e-shared-a-w3-43d592-54c9b96dcf-6bwjj\" (UID: \"0c63b14b-1649-4c30-b788-81e4e75e40f4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-43d592-54c9b96dcf-6bwjj"
Oct 30 22:29:08.208267 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.204428    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c63b14b-1649-4c30-b788-81e4e75e40f4-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-43d592-54c9b96dcf-6bwjj\" (UID: \"0c63b14b-1649-4c30-b788-81e4e75e40f4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-43d592-54c9b96dcf-6bwjj"
Oct 30 22:29:08.208267 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.206698    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-43d592-dockercfg-k6zn4\""
Oct 30 22:29:08.208267 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.206783    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-43d592\""
Oct 30 22:29:08.299595 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.299037    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"
Oct 30 22:29:08.299595 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.299169    4871 patch_prober.go:29] interesting pod/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.176:15021/healthz/ready\": dial tcp 10.133.0.176:15021: connect: connection refused" start-of-body=
Oct 30 22:29:08.299595 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.299212    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp" podUID="6cbecab8-bfad-40bf-850a-905e4d365d29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.176:15021/healthz/ready\": dial tcp 10.133.0.176:15021: connect: connection refused"
Oct 30 22:29:08.323444 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.307033    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c63b14b-1649-4c30-b788-81e4e75e40f4-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-43d592-54c9b96dcf-6bwjj\" (UID: \"0c63b14b-1649-4c30-b788-81e4e75e40f4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-43d592-54c9b96dcf-6bwjj"
Oct 30 22:29:08.323444 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.307318    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8gzw\" (UniqueName: \"kubernetes.io/projected/0c63b14b-1649-4c30-b788-81e4e75e40f4-kube-api-access-h8gzw\") pod \"payload-pre-processing-e2e-shared-a-w3-43d592-54c9b96dcf-6bwjj\" (UID: \"0c63b14b-1649-4c30-b788-81e4e75e40f4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-43d592-54c9b96dcf-6bwjj"
Oct 30 22:29:08.324347 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.324228    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c63b14b-1649-4c30-b788-81e4e75e40f4-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-43d592-54c9b96dcf-6bwjj\" (UID: \"0c63b14b-1649-4c30-b788-81e4e75e40f4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-43d592-54c9b96dcf-6bwjj"
Oct 30 22:29:08.402294 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.388297    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-43d592-6cf85b6645-f4dvl"]
Oct 30 22:29:08.418263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.408080    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-43d592-6cf85b6645-f4dvl"
Oct 30 22:29:08.426063 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.425989    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8gzw\" (UniqueName: \"kubernetes.io/projected/0c63b14b-1649-4c30-b788-81e4e75e40f4-kube-api-access-h8gzw\") pod \"payload-pre-processing-e2e-shared-a-w3-43d592-54c9b96dcf-6bwjj\" (UID: \"0c63b14b-1649-4c30-b788-81e4e75e40f4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-43d592-54c9b96dcf-6bwjj"
Oct 30 22:29:08.472903 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.472865    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-43d592-54c9b96dcf-6bwjj"
Oct 30 22:29:08.477902 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.477867    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-43d592-6cf85b6645-f4dvl"]
Oct 30 22:29:08.520330 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.520286    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-994tf\" (UniqueName: \"kubernetes.io/projected/3ab9ef68-5a6d-495f-be71-68366e8ab27d-kube-api-access-994tf\") pod \"payload-processing-e2e-shared-a-w3-43d592-6cf85b6645-f4dvl\" (UID: \"3ab9ef68-5a6d-495f-be71-68366e8ab27d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-43d592-6cf85b6645-f4dvl"
Oct 30 22:29:08.520526 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.520458    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ab9ef68-5a6d-495f-be71-68366e8ab27d-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-43d592-6cf85b6645-f4dvl\" (UID: \"3ab9ef68-5a6d-495f-be71-68366e8ab27d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-43d592-6cf85b6645-f4dvl"
Oct 30 22:29:08.632228 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.630467    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-994tf\" (UniqueName: \"kubernetes.io/projected/3ab9ef68-5a6d-495f-be71-68366e8ab27d-kube-api-access-994tf\") pod \"payload-processing-e2e-shared-a-w3-43d592-6cf85b6645-f4dvl\" (UID: \"3ab9ef68-5a6d-495f-be71-68366e8ab27d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-43d592-6cf85b6645-f4dvl"
Oct 30 22:29:08.632228 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.630682    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ab9ef68-5a6d-495f-be71-68366e8ab27d-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-43d592-6cf85b6645-f4dvl\" (UID: \"3ab9ef68-5a6d-495f-be71-68366e8ab27d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-43d592-6cf85b6645-f4dvl"
Oct 30 22:29:08.632228 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.631733    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ab9ef68-5a6d-495f-be71-68366e8ab27d-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-43d592-6cf85b6645-f4dvl\" (UID: \"3ab9ef68-5a6d-495f-be71-68366e8ab27d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-43d592-6cf85b6645-f4dvl"
Oct 30 22:29:08.696156 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.696046    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-994tf\" (UniqueName: \"kubernetes.io/projected/3ab9ef68-5a6d-495f-be71-68366e8ab27d-kube-api-access-994tf\") pod \"payload-processing-e2e-shared-a-w3-43d592-6cf85b6645-f4dvl\" (UID: \"3ab9ef68-5a6d-495f-be71-68366e8ab27d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-43d592-6cf85b6645-f4dvl"
Oct 30 22:29:08.756373 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.756338    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-43d592-6cf85b6645-f4dvl"
Oct 30 22:29:08.783458 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.783412    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"]
Oct 30 22:29:08.800668 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.800632    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"
Oct 30 22:29:08.867317 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.835983    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/478b54f8-6c41-42ad-ac53-cb6144f07773-istio-podinfo\") pod \"e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2\" (UID: \"478b54f8-6c41-42ad-ac53-cb6144f07773\") " pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"
Oct 30 22:29:08.867317 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.836058    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/478b54f8-6c41-42ad-ac53-cb6144f07773-istio-envoy\") pod \"e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2\" (UID: \"478b54f8-6c41-42ad-ac53-cb6144f07773\") " pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"
Oct 30 22:29:08.867317 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.836121    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/478b54f8-6c41-42ad-ac53-cb6144f07773-istio-token\") pod \"e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2\" (UID: \"478b54f8-6c41-42ad-ac53-cb6144f07773\") " pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"
Oct 30 22:29:08.867317 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.836170    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/478b54f8-6c41-42ad-ac53-cb6144f07773-istio-data\") pod \"e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2\" (UID: \"478b54f8-6c41-42ad-ac53-cb6144f07773\") " pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"
Oct 30 22:29:08.867317 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.836207    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/478b54f8-6c41-42ad-ac53-cb6144f07773-credential-socket\") pod \"e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2\" (UID: \"478b54f8-6c41-42ad-ac53-cb6144f07773\") " pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"
Oct 30 22:29:08.867317 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.836257    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/478b54f8-6c41-42ad-ac53-cb6144f07773-workload-socket\") pod \"e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2\" (UID: \"478b54f8-6c41-42ad-ac53-cb6144f07773\") " pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"
Oct 30 22:29:08.867317 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.836312    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95d95\" (UniqueName: \"kubernetes.io/projected/478b54f8-6c41-42ad-ac53-cb6144f07773-kube-api-access-95d95\") pod \"e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2\" (UID: \"478b54f8-6c41-42ad-ac53-cb6144f07773\") " pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"
Oct 30 22:29:08.867317 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.836417    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/478b54f8-6c41-42ad-ac53-cb6144f07773-workload-certs\") pod \"e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2\" (UID: \"478b54f8-6c41-42ad-ac53-cb6144f07773\") " pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"
Oct 30 22:29:08.867317 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.836445    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/478b54f8-6c41-42ad-ac53-cb6144f07773-istiod-ca-cert\") pod \"e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2\" (UID: \"478b54f8-6c41-42ad-ac53-cb6144f07773\") " pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"
Oct 30 22:29:08.955625 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.937559    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/478b54f8-6c41-42ad-ac53-cb6144f07773-istio-data\") pod \"e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2\" (UID: \"478b54f8-6c41-42ad-ac53-cb6144f07773\") " pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"
Oct 30 22:29:08.955625 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.937618    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/478b54f8-6c41-42ad-ac53-cb6144f07773-credential-socket\") pod \"e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2\" (UID: \"478b54f8-6c41-42ad-ac53-cb6144f07773\") " pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"
Oct 30 22:29:08.955625 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.937648    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/478b54f8-6c41-42ad-ac53-cb6144f07773-workload-socket\") pod \"e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2\" (UID: \"478b54f8-6c41-42ad-ac53-cb6144f07773\") " pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"
Oct 30 22:29:08.955625 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.937700    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95d95\" (UniqueName: \"kubernetes.io/projected/478b54f8-6c41-42ad-ac53-cb6144f07773-kube-api-access-95d95\") pod \"e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2\" (UID: \"478b54f8-6c41-42ad-ac53-cb6144f07773\") " pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"
Oct 30 22:29:08.955625 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.937796    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/478b54f8-6c41-42ad-ac53-cb6144f07773-workload-certs\") pod \"e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2\" (UID: \"478b54f8-6c41-42ad-ac53-cb6144f07773\") " pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"
Oct 30 22:29:08.955625 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.937824    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/478b54f8-6c41-42ad-ac53-cb6144f07773-istiod-ca-cert\") pod \"e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2\" (UID: \"478b54f8-6c41-42ad-ac53-cb6144f07773\") " pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"
Oct 30 22:29:08.955625 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.937920    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/478b54f8-6c41-42ad-ac53-cb6144f07773-istio-podinfo\") pod \"e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2\" (UID: \"478b54f8-6c41-42ad-ac53-cb6144f07773\") " pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"
Oct 30 22:29:08.955625 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.937965    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/478b54f8-6c41-42ad-ac53-cb6144f07773-istio-envoy\") pod \"e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2\" (UID: \"478b54f8-6c41-42ad-ac53-cb6144f07773\") " pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"
Oct 30 22:29:08.955625 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.938016    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/478b54f8-6c41-42ad-ac53-cb6144f07773-istio-token\") pod \"e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2\" (UID: \"478b54f8-6c41-42ad-ac53-cb6144f07773\") " pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"
Oct 30 22:29:08.955625 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.938912    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/478b54f8-6c41-42ad-ac53-cb6144f07773-istio-data\") pod \"e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2\" (UID: \"478b54f8-6c41-42ad-ac53-cb6144f07773\") " pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"
Oct 30 22:29:08.955625 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.939152    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/478b54f8-6c41-42ad-ac53-cb6144f07773-credential-socket\") pod \"e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2\" (UID: \"478b54f8-6c41-42ad-ac53-cb6144f07773\") " pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"
Oct 30 22:29:08.955625 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.939402    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/478b54f8-6c41-42ad-ac53-cb6144f07773-workload-socket\") pod \"e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2\" (UID: \"478b54f8-6c41-42ad-ac53-cb6144f07773\") " pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"
Oct 30 22:29:08.955625 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.939879    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/478b54f8-6c41-42ad-ac53-cb6144f07773-workload-certs\") pod \"e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2\" (UID: \"478b54f8-6c41-42ad-ac53-cb6144f07773\") " pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"
Oct 30 22:29:08.986428 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.964438    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:08.986428 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:08.964806    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w0-6fc60c-openshift-default-dockercfg-jjdhz\""
Oct 30 22:29:09.056299 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:09.049497    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/478b54f8-6c41-42ad-ac53-cb6144f07773-istiod-ca-cert\") pod \"e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2\" (UID: \"478b54f8-6c41-42ad-ac53-cb6144f07773\") " pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"
Oct 30 22:29:09.086319 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:09.081426    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"]
Oct 30 22:29:09.106254 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:09.104449    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-95d95\" (UniqueName: \"kubernetes.io/projected/478b54f8-6c41-42ad-ac53-cb6144f07773-kube-api-access-95d95\") pod \"e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2\" (UID: \"478b54f8-6c41-42ad-ac53-cb6144f07773\") " pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"
Oct 30 22:29:09.106254 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:29:09.104725    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c63b14b_1649_4c30_b788_81e4e75e40f4.slice/crio-28d32bc50dc2b50132d8fe2d9436125c60ba5ec99975ed8829073d26c95c17f5 WatchSource:0}: Error finding container 28d32bc50dc2b50132d8fe2d9436125c60ba5ec99975ed8829073d26c95c17f5: Status 404 returned error can't find the container with id 28d32bc50dc2b50132d8fe2d9436125c60ba5ec99975ed8829073d26c95c17f5
Oct 30 22:29:09.108402 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:09.108114    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/478b54f8-6c41-42ad-ac53-cb6144f07773-istio-token\") pod \"e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2\" (UID: \"478b54f8-6c41-42ad-ac53-cb6144f07773\") " pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"
Oct 30 22:29:09.112866 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:09.112462    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-43d592-54c9b96dcf-6bwjj"]
Oct 30 22:29:09.146672 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:09.138496    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/478b54f8-6c41-42ad-ac53-cb6144f07773-istio-envoy\") pod \"e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2\" (UID: \"478b54f8-6c41-42ad-ac53-cb6144f07773\") " pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"
Oct 30 22:29:09.156806 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:09.155799    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/478b54f8-6c41-42ad-ac53-cb6144f07773-istio-podinfo\") pod \"e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2\" (UID: \"478b54f8-6c41-42ad-ac53-cb6144f07773\") " pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"
Oct 30 22:29:09.191410 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:09.179120    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"
Oct 30 22:29:09.191410 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:09.181605    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:09.294335 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:09.294304    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"
Oct 30 22:29:09.341922 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:09.341885    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:09.448883 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:29:09.448845    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ab9ef68_5a6d_495f_be71_68366e8ab27d.slice/crio-99653443e44ccfc8c60e4b8628303f7b83c4df476ac1ae56b83e7e28b4e0b592 WatchSource:0}: Error finding container 99653443e44ccfc8c60e4b8628303f7b83c4df476ac1ae56b83e7e28b4e0b592: Status 404 returned error can't find the container with id 99653443e44ccfc8c60e4b8628303f7b83c4df476ac1ae56b83e7e28b4e0b592
Oct 30 22:29:09.454088 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:09.454057    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-43d592-6cf85b6645-f4dvl"]
Oct 30 22:29:09.617262 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:29:09.617191    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod478b54f8_6c41_42ad_ac53_cb6144f07773.slice/crio-aed50fe150b503675954880fbe3f93463d10582d970d4022478542ec7833596f WatchSource:0}: Error finding container aed50fe150b503675954880fbe3f93463d10582d970d4022478542ec7833596f: Status 404 returned error can't find the container with id aed50fe150b503675954880fbe3f93463d10582d970d4022478542ec7833596f
Oct 30 22:29:09.620086 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:09.620052    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:29:09.620207 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:09.620123    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:29:09.620207 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:09.620156    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:29:09.624477 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:09.624449    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"]
Oct 30 22:29:09.800300 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:09.800103    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:10.177561 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:10.177522    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-43d592-54c9b96dcf-6bwjj" event={"ID":"0c63b14b-1649-4c30-b788-81e4e75e40f4","Type":"ContainerStarted","Data":"67ef3c2eb139be350d421fa060a4ad9dee505758429b6b98f847e244052d5bd2"}
Oct 30 22:29:10.178076 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:10.177587    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-43d592-54c9b96dcf-6bwjj"
Oct 30 22:29:10.178076 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:10.177612    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-43d592-54c9b96dcf-6bwjj" event={"ID":"0c63b14b-1649-4c30-b788-81e4e75e40f4","Type":"ContainerStarted","Data":"28d32bc50dc2b50132d8fe2d9436125c60ba5ec99975ed8829073d26c95c17f5"}
Oct 30 22:29:10.179511 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:10.179483    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2" event={"ID":"478b54f8-6c41-42ad-ac53-cb6144f07773","Type":"ContainerStarted","Data":"f6f414d401e6b5799505fff1643916daf0a5f8c0be737f58c1484a5a7b96a6bc"}
Oct 30 22:29:10.179638 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:10.179533    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2" event={"ID":"478b54f8-6c41-42ad-ac53-cb6144f07773","Type":"ContainerStarted","Data":"aed50fe150b503675954880fbe3f93463d10582d970d4022478542ec7833596f"}
Oct 30 22:29:10.180021 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:10.179996    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"
Oct 30 22:29:10.181288 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:10.181262    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-43d592-6cf85b6645-f4dvl" event={"ID":"3ab9ef68-5a6d-495f-be71-68366e8ab27d","Type":"ContainerStarted","Data":"582e68a8923e599b6b0503f83367e54abb24ba48d9d41d13a8849fc1485a6f90"}
Oct 30 22:29:10.181398 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:10.181308    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-43d592-6cf85b6645-f4dvl" event={"ID":"3ab9ef68-5a6d-495f-be71-68366e8ab27d","Type":"ContainerStarted","Data":"99653443e44ccfc8c60e4b8628303f7b83c4df476ac1ae56b83e7e28b4e0b592"}
Oct 30 22:29:10.181570 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:10.181541    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-43d592-6cf85b6645-f4dvl"
Oct 30 22:29:10.181683 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:10.181587    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"
Oct 30 22:29:10.182599 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:10.182581    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"
Oct 30 22:29:10.185400 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:10.185381    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"
Oct 30 22:29:10.302479 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:10.302420    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-43d592-54c9b96dcf-6bwjj" podStartSLOduration=2.302405576 podStartE2EDuration="2.302405576s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:29:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:29:10.298045245 +0000 UTC m=+3092.475533862" watchObservedRunningTime="2026-10-30 22:29:10.302405576 +0000 UTC m=+3092.479894197"
Oct 30 22:29:10.448313 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:10.447409    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-43d592-6cf85b6645-f4dvl" podStartSLOduration=2.447386335 podStartE2EDuration="2.447386335s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:29:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:29:10.430458355 +0000 UTC m=+3092.607947004" watchObservedRunningTime="2026-10-30 22:29:10.447386335 +0000 UTC m=+3092.624874950"
Oct 30 22:29:10.586010 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:10.585916    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2" podStartSLOduration=2.585891361 podStartE2EDuration="2.585891361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:29:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:29:10.562608885 +0000 UTC m=+3092.740097541" watchObservedRunningTime="2026-10-30 22:29:10.585891361 +0000 UTC m=+3092.763379976"
Oct 30 22:29:11.190149 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:11.190111    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"
Oct 30 22:29:11.190866 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:11.190850    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"
Oct 30 22:29:13.250207 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:13.250173    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:13.424934 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:13.424896    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:13.586297 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:13.586057    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:14.795171 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:14.795133    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:16.466713 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:16.466667    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:16.667196 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:16.667164    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:19.581649 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:19.581567    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:20.849143 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:20.849109    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:21.021874 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:21.021836    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:21.192435 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:21.192402    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-43d592-6cf85b6645-f4dvl"
Oct 30 22:29:21.192623 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:21.192480    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-43d592-54c9b96dcf-6bwjj"
Oct 30 22:29:21.208037 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:21.208001    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:24.101681 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:24.101627    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:24.297417 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:24.297225    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:24.931573 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:24.931535    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"]
Oct 30 22:29:24.937013 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:24.936981    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"
Oct 30 22:29:24.999294 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:24.999217    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"]
Oct 30 22:29:25.038609 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.038564    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:25.042280 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.042229    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4886da1-54b1-4252-bb65-553e0ec54d7a-workload-socket\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q\" (UID: \"c4886da1-54b1-4252-bb65-553e0ec54d7a\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"
Oct 30 22:29:25.042450 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.042307    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4886da1-54b1-4252-bb65-553e0ec54d7a-istio-envoy\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q\" (UID: \"c4886da1-54b1-4252-bb65-553e0ec54d7a\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"
Oct 30 22:29:25.042450 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.042351    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4886da1-54b1-4252-bb65-553e0ec54d7a-istiod-ca-cert\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q\" (UID: \"c4886da1-54b1-4252-bb65-553e0ec54d7a\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"
Oct 30 22:29:25.042450 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.042409    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4886da1-54b1-4252-bb65-553e0ec54d7a-istio-podinfo\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q\" (UID: \"c4886da1-54b1-4252-bb65-553e0ec54d7a\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"
Oct 30 22:29:25.042627 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.042506    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4886da1-54b1-4252-bb65-553e0ec54d7a-istio-token\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q\" (UID: \"c4886da1-54b1-4252-bb65-553e0ec54d7a\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"
Oct 30 22:29:25.042627 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.042572    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jg65z\" (UniqueName: \"kubernetes.io/projected/c4886da1-54b1-4252-bb65-553e0ec54d7a-kube-api-access-jg65z\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q\" (UID: \"c4886da1-54b1-4252-bb65-553e0ec54d7a\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"
Oct 30 22:29:25.042755 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.042645    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4886da1-54b1-4252-bb65-553e0ec54d7a-istio-data\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q\" (UID: \"c4886da1-54b1-4252-bb65-553e0ec54d7a\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"
Oct 30 22:29:25.042755 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.042695    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4886da1-54b1-4252-bb65-553e0ec54d7a-credential-socket\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q\" (UID: \"c4886da1-54b1-4252-bb65-553e0ec54d7a\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"
Oct 30 22:29:25.042868 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.042832    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4886da1-54b1-4252-bb65-553e0ec54d7a-workload-certs\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q\" (UID: \"c4886da1-54b1-4252-bb65-553e0ec54d7a\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"
Oct 30 22:29:25.045996 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.045967    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-a-f7c488-openshift-default-dockercfg-2kvn7\""
Oct 30 22:29:25.144065 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.144029    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4886da1-54b1-4252-bb65-553e0ec54d7a-workload-certs\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q\" (UID: \"c4886da1-54b1-4252-bb65-553e0ec54d7a\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"
Oct 30 22:29:25.144576 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.144164    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4886da1-54b1-4252-bb65-553e0ec54d7a-workload-socket\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q\" (UID: \"c4886da1-54b1-4252-bb65-553e0ec54d7a\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"
Oct 30 22:29:25.144576 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.144190    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4886da1-54b1-4252-bb65-553e0ec54d7a-istio-envoy\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q\" (UID: \"c4886da1-54b1-4252-bb65-553e0ec54d7a\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"
Oct 30 22:29:25.144576 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.144213    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4886da1-54b1-4252-bb65-553e0ec54d7a-istiod-ca-cert\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q\" (UID: \"c4886da1-54b1-4252-bb65-553e0ec54d7a\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"
Oct 30 22:29:25.144576 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.144255    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4886da1-54b1-4252-bb65-553e0ec54d7a-istio-podinfo\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q\" (UID: \"c4886da1-54b1-4252-bb65-553e0ec54d7a\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"
Oct 30 22:29:25.144576 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.144284    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4886da1-54b1-4252-bb65-553e0ec54d7a-istio-token\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q\" (UID: \"c4886da1-54b1-4252-bb65-553e0ec54d7a\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"
Oct 30 22:29:25.144576 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.144342    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jg65z\" (UniqueName: \"kubernetes.io/projected/c4886da1-54b1-4252-bb65-553e0ec54d7a-kube-api-access-jg65z\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q\" (UID: \"c4886da1-54b1-4252-bb65-553e0ec54d7a\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"
Oct 30 22:29:25.144576 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.144384    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4886da1-54b1-4252-bb65-553e0ec54d7a-istio-data\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q\" (UID: \"c4886da1-54b1-4252-bb65-553e0ec54d7a\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"
Oct 30 22:29:25.144576 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.144443    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4886da1-54b1-4252-bb65-553e0ec54d7a-credential-socket\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q\" (UID: \"c4886da1-54b1-4252-bb65-553e0ec54d7a\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"
Oct 30 22:29:25.144576 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.144564    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4886da1-54b1-4252-bb65-553e0ec54d7a-workload-certs\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q\" (UID: \"c4886da1-54b1-4252-bb65-553e0ec54d7a\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"
Oct 30 22:29:25.145099 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.144846    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4886da1-54b1-4252-bb65-553e0ec54d7a-istio-data\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q\" (UID: \"c4886da1-54b1-4252-bb65-553e0ec54d7a\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"
Oct 30 22:29:25.145099 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.145080    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4886da1-54b1-4252-bb65-553e0ec54d7a-istiod-ca-cert\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q\" (UID: \"c4886da1-54b1-4252-bb65-553e0ec54d7a\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"
Oct 30 22:29:25.145580 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.145557    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4886da1-54b1-4252-bb65-553e0ec54d7a-workload-socket\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q\" (UID: \"c4886da1-54b1-4252-bb65-553e0ec54d7a\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"
Oct 30 22:29:25.145703 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.145680    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4886da1-54b1-4252-bb65-553e0ec54d7a-credential-socket\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q\" (UID: \"c4886da1-54b1-4252-bb65-553e0ec54d7a\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"
Oct 30 22:29:25.147226 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.147201    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4886da1-54b1-4252-bb65-553e0ec54d7a-istio-envoy\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q\" (UID: \"c4886da1-54b1-4252-bb65-553e0ec54d7a\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"
Oct 30 22:29:25.147713 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.147688    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4886da1-54b1-4252-bb65-553e0ec54d7a-istio-podinfo\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q\" (UID: \"c4886da1-54b1-4252-bb65-553e0ec54d7a\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"
Oct 30 22:29:25.210470 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.210385    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4886da1-54b1-4252-bb65-553e0ec54d7a-istio-token\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q\" (UID: \"c4886da1-54b1-4252-bb65-553e0ec54d7a\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"
Oct 30 22:29:25.210622 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.210512    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jg65z\" (UniqueName: \"kubernetes.io/projected/c4886da1-54b1-4252-bb65-553e0ec54d7a-kube-api-access-jg65z\") pod \"e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q\" (UID: \"c4886da1-54b1-4252-bb65-553e0ec54d7a\") " pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"
Oct 30 22:29:25.258416 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.258375    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"
Oct 30 22:29:25.690127 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.690078    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"]
Oct 30 22:29:25.695396 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:29:25.695362    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4886da1_54b1_4252_bb65_553e0ec54d7a.slice/crio-cf09da268cdbbc118abcf5ff537be9c6a90972828b54ce64c29aa4750208f621 WatchSource:0}: Error finding container cf09da268cdbbc118abcf5ff537be9c6a90972828b54ce64c29aa4750208f621: Status 404 returned error can't find the container with id cf09da268cdbbc118abcf5ff537be9c6a90972828b54ce64c29aa4750208f621
Oct 30 22:29:25.696649 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.696629    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"
Oct 30 22:29:25.698291 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.698251    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:29:25.698396 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.698334    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:29:25.698396 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.698365    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:29:25.701365 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.701340    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"]
Oct 30 22:29:25.741753 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.741721    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"]
Oct 30 22:29:25.749850 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.749819    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"
Oct 30 22:29:25.753626 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.753598    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"]
Oct 30 22:29:25.847035 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.845345    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"]
Oct 30 22:29:25.856159 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.855384    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a69cc06-f8e1-499a-828d-9876d05ed442-istio-podinfo\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2\" (UID: \"0a69cc06-f8e1-499a-828d-9876d05ed442\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"
Oct 30 22:29:25.856159 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.855478    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a69cc06-f8e1-499a-828d-9876d05ed442-credential-socket\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2\" (UID: \"0a69cc06-f8e1-499a-828d-9876d05ed442\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"
Oct 30 22:29:25.856159 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.855510    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-istio-data\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds\" (UID: \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"
Oct 30 22:29:25.856159 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.855542    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-credential-socket\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds\" (UID: \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"
Oct 30 22:29:25.856159 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.855576    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-workload-certs\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds\" (UID: \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"
Oct 30 22:29:25.856159 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.855600    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtrjj\" (UniqueName: \"kubernetes.io/projected/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-kube-api-access-xtrjj\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds\" (UID: \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"
Oct 30 22:29:25.856159 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.855653    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a69cc06-f8e1-499a-828d-9876d05ed442-workload-socket\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2\" (UID: \"0a69cc06-f8e1-499a-828d-9876d05ed442\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"
Oct 30 22:29:25.856159 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.855714    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-istiod-ca-cert\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds\" (UID: \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"
Oct 30 22:29:25.856159 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.855759    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-istio-token\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds\" (UID: \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"
Oct 30 22:29:25.856159 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.855804    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8x9l9\" (UniqueName: \"kubernetes.io/projected/0a69cc06-f8e1-499a-828d-9876d05ed442-kube-api-access-8x9l9\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2\" (UID: \"0a69cc06-f8e1-499a-828d-9876d05ed442\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"
Oct 30 22:29:25.856159 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.855828    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a69cc06-f8e1-499a-828d-9876d05ed442-istiod-ca-cert\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2\" (UID: \"0a69cc06-f8e1-499a-828d-9876d05ed442\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"
Oct 30 22:29:25.856159 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.855882    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a69cc06-f8e1-499a-828d-9876d05ed442-istio-data\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2\" (UID: \"0a69cc06-f8e1-499a-828d-9876d05ed442\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"
Oct 30 22:29:25.856159 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.855950    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a69cc06-f8e1-499a-828d-9876d05ed442-istio-token\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2\" (UID: \"0a69cc06-f8e1-499a-828d-9876d05ed442\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"
Oct 30 22:29:25.856159 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.855973    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-workload-socket\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds\" (UID: \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"
Oct 30 22:29:25.856159 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.855997    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-istio-envoy\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds\" (UID: \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"
Oct 30 22:29:25.856159 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.856034    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a69cc06-f8e1-499a-828d-9876d05ed442-workload-certs\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2\" (UID: \"0a69cc06-f8e1-499a-828d-9876d05ed442\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"
Oct 30 22:29:25.856159 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.856073    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-istio-podinfo\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds\" (UID: \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"
Oct 30 22:29:25.858738 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.856102    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a69cc06-f8e1-499a-828d-9876d05ed442-istio-envoy\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2\" (UID: \"0a69cc06-f8e1-499a-828d-9876d05ed442\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"
Oct 30 22:29:25.960346 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.960253    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a69cc06-f8e1-499a-828d-9876d05ed442-istio-data\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2\" (UID: \"0a69cc06-f8e1-499a-828d-9876d05ed442\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"
Oct 30 22:29:25.960346 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.960319    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a69cc06-f8e1-499a-828d-9876d05ed442-istio-token\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2\" (UID: \"0a69cc06-f8e1-499a-828d-9876d05ed442\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"
Oct 30 22:29:25.960346 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.960339    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-workload-socket\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds\" (UID: \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"
Oct 30 22:29:25.960638 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.960364    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-istio-envoy\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds\" (UID: \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"
Oct 30 22:29:25.960638 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.960387    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a69cc06-f8e1-499a-828d-9876d05ed442-workload-certs\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2\" (UID: \"0a69cc06-f8e1-499a-828d-9876d05ed442\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"
Oct 30 22:29:25.960638 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.960412    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-istio-podinfo\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds\" (UID: \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"
Oct 30 22:29:25.960863 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.960687    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a69cc06-f8e1-499a-828d-9876d05ed442-istio-data\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2\" (UID: \"0a69cc06-f8e1-499a-828d-9876d05ed442\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"
Oct 30 22:29:25.960863 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.960756    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a69cc06-f8e1-499a-828d-9876d05ed442-workload-certs\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2\" (UID: \"0a69cc06-f8e1-499a-828d-9876d05ed442\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"
Oct 30 22:29:25.960863 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.960792    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-workload-socket\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds\" (UID: \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"
Oct 30 22:29:25.963324 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.963293    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a69cc06-f8e1-499a-828d-9876d05ed442-istio-envoy\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2\" (UID: \"0a69cc06-f8e1-499a-828d-9876d05ed442\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"
Oct 30 22:29:25.963452 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.963365    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a69cc06-f8e1-499a-828d-9876d05ed442-istio-podinfo\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2\" (UID: \"0a69cc06-f8e1-499a-828d-9876d05ed442\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"
Oct 30 22:29:25.963452 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.963373    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a69cc06-f8e1-499a-828d-9876d05ed442-istio-envoy\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2\" (UID: \"0a69cc06-f8e1-499a-828d-9876d05ed442\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"
Oct 30 22:29:25.963452 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.963431    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a69cc06-f8e1-499a-828d-9876d05ed442-credential-socket\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2\" (UID: \"0a69cc06-f8e1-499a-828d-9876d05ed442\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"
Oct 30 22:29:25.963625 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.963457    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-istio-data\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds\" (UID: \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"
Oct 30 22:29:25.963625 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.963491    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-credential-socket\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds\" (UID: \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"
Oct 30 22:29:25.963625 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.963452    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-istio-envoy\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds\" (UID: \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"
Oct 30 22:29:25.963625 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.963514    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-workload-certs\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds\" (UID: \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"
Oct 30 22:29:25.963625 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.963544    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtrjj\" (UniqueName: \"kubernetes.io/projected/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-kube-api-access-xtrjj\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds\" (UID: \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"
Oct 30 22:29:25.963625 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.963565    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-istio-podinfo\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds\" (UID: \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"
Oct 30 22:29:25.963625 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.963620    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a69cc06-f8e1-499a-828d-9876d05ed442-workload-socket\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2\" (UID: \"0a69cc06-f8e1-499a-828d-9876d05ed442\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"
Oct 30 22:29:25.964007 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.963674    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a69cc06-f8e1-499a-828d-9876d05ed442-credential-socket\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2\" (UID: \"0a69cc06-f8e1-499a-828d-9876d05ed442\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"
Oct 30 22:29:25.964007 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.963697    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-istiod-ca-cert\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds\" (UID: \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"
Oct 30 22:29:25.964007 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.963705    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-istio-data\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds\" (UID: \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"
Oct 30 22:29:25.964007 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.963763    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-istio-token\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds\" (UID: \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"
Oct 30 22:29:25.964007 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.963810    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x9l9\" (UniqueName: \"kubernetes.io/projected/0a69cc06-f8e1-499a-828d-9876d05ed442-kube-api-access-8x9l9\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2\" (UID: \"0a69cc06-f8e1-499a-828d-9876d05ed442\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"
Oct 30 22:29:25.964007 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.963836    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a69cc06-f8e1-499a-828d-9876d05ed442-istiod-ca-cert\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2\" (UID: \"0a69cc06-f8e1-499a-828d-9876d05ed442\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"
Oct 30 22:29:25.964007 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.963893    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-credential-socket\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds\" (UID: \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"
Oct 30 22:29:25.964007 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.963919    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-workload-certs\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds\" (UID: \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"
Oct 30 22:29:25.964007 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.963948    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a69cc06-f8e1-499a-828d-9876d05ed442-workload-socket\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2\" (UID: \"0a69cc06-f8e1-499a-828d-9876d05ed442\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"
Oct 30 22:29:25.964529 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.964324    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-istiod-ca-cert\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds\" (UID: \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"
Oct 30 22:29:25.964529 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.964476    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a69cc06-f8e1-499a-828d-9876d05ed442-istiod-ca-cert\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2\" (UID: \"0a69cc06-f8e1-499a-828d-9876d05ed442\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"
Oct 30 22:29:25.965730 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:25.965709    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a69cc06-f8e1-499a-828d-9876d05ed442-istio-podinfo\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2\" (UID: \"0a69cc06-f8e1-499a-828d-9876d05ed442\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"
Oct 30 22:29:26.027132 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:26.027072    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a69cc06-f8e1-499a-828d-9876d05ed442-istio-token\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2\" (UID: \"0a69cc06-f8e1-499a-828d-9876d05ed442\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"
Oct 30 22:29:26.028013 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:26.027977    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8x9l9\" (UniqueName: \"kubernetes.io/projected/0a69cc06-f8e1-499a-828d-9876d05ed442-kube-api-access-8x9l9\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2\" (UID: \"0a69cc06-f8e1-499a-828d-9876d05ed442\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"
Oct 30 22:29:26.028293 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:26.028273    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-istio-token\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds\" (UID: \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"
Oct 30 22:29:26.028638 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:26.028612    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtrjj\" (UniqueName: \"kubernetes.io/projected/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-kube-api-access-xtrjj\") pod \"e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds\" (UID: \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\") " pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"
Oct 30 22:29:26.081661 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:26.081614    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"
Oct 30 22:29:26.315094 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:26.315012    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"
Oct 30 22:29:26.345172 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:26.345140    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q" event={"ID":"c4886da1-54b1-4252-bb65-553e0ec54d7a","Type":"ContainerStarted","Data":"eb80ca2751b43e6654c61c4fd592440957b2f768eea10ab1dd01bbb8fce304ee"}
Oct 30 22:29:26.345341 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:26.345233    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q" event={"ID":"c4886da1-54b1-4252-bb65-553e0ec54d7a","Type":"ContainerStarted","Data":"cf09da268cdbbc118abcf5ff537be9c6a90972828b54ce64c29aa4750208f621"}
Oct 30 22:29:26.512272 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:26.510581    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q" podStartSLOduration=2.510558758 podStartE2EDuration="2.510558758s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:29:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:29:26.507613137 +0000 UTC m=+3108.685101754" watchObservedRunningTime="2026-10-30 22:29:26.510558758 +0000 UTC m=+3108.688047377"
Oct 30 22:29:26.543054 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:26.542407    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"]
Oct 30 22:29:26.554953 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:26.554902    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:29:26.555092 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:26.555016    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:29:26.555092 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:26.555067    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:29:26.771214 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:29:26.771183    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a69cc06_f8e1_499a_828d_9876d05ed442.slice/crio-4ac8064ca4906ee48c28a46d381950d22058e2a6363b944254b33a20b3c4d1de WatchSource:0}: Error finding container 4ac8064ca4906ee48c28a46d381950d22058e2a6363b944254b33a20b3c4d1de: Status 404 returned error can't find the container with id 4ac8064ca4906ee48c28a46d381950d22058e2a6363b944254b33a20b3c4d1de
Oct 30 22:29:26.773756 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:26.773712    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:29:26.773867 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:26.773838    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:29:26.773923 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:26.773877    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:29:26.774534 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:26.774484    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"]
Oct 30 22:29:27.260812 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:27.260720    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"
Oct 30 22:29:27.261001 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:27.260825    4871 patch_prober.go:29] interesting pod/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.180:15021/healthz/ready\": dial tcp 10.133.0.180:15021: connect: connection refused" start-of-body=
Oct 30 22:29:27.261001 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:27.260863    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q" podUID="c4886da1-54b1-4252-bb65-553e0ec54d7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.180:15021/healthz/ready\": dial tcp 10.133.0.180:15021: connect: connection refused"
Oct 30 22:29:27.357677 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:27.356829    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2" event={"ID":"0a69cc06-f8e1-499a-828d-9876d05ed442","Type":"ContainerStarted","Data":"f145a98c5286f1dd284fada0f395a6a677fa50e96fee6b183b8da091a3a7b5f8"}
Oct 30 22:29:27.357677 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:27.356906    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2" event={"ID":"0a69cc06-f8e1-499a-828d-9876d05ed442","Type":"ContainerStarted","Data":"4ac8064ca4906ee48c28a46d381950d22058e2a6363b944254b33a20b3c4d1de"}
Oct 30 22:29:27.359718 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:27.359049    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds" event={"ID":"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11","Type":"ContainerStarted","Data":"07359f5c86dbdc876784add945ce17fc9546b05db8fcaceb7db7005f886c0c7d"}
Oct 30 22:29:27.359718 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:27.359117    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds" event={"ID":"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11","Type":"ContainerStarted","Data":"abe1d16ccde714ad43f830e3910e5f78e261a047849c2e9577564feed13eab59"}
Oct 30 22:29:27.497778 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:27.497718    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2" podStartSLOduration=2.497702512 podStartE2EDuration="2.497702512s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:29:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:29:27.493935622 +0000 UTC m=+3109.671424236" watchObservedRunningTime="2026-10-30 22:29:27.497702512 +0000 UTC m=+3109.675191129"
Oct 30 22:29:27.641260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:27.641118    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds" podStartSLOduration=2.641095408 podStartE2EDuration="2.641095408s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:29:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:29:27.62964609 +0000 UTC m=+3109.807134719" watchObservedRunningTime="2026-10-30 22:29:27.641095408 +0000 UTC m=+3109.818584024"
Oct 30 22:29:28.084700 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:28.084598    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"
Oct 30 22:29:28.085052 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:28.085027    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.182:15021/healthz/ready\": dial tcp 10.133.0.182:15021: connect: connection refused" start-of-body=
Oct 30 22:29:28.085222 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:28.085196    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds" podUID="eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.182:15021/healthz/ready\": dial tcp 10.133.0.182:15021: connect: connection refused"
Oct 30 22:29:28.266060 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:28.266021    4871 patch_prober.go:29] interesting pod/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.180:15021/healthz/ready\": dial tcp 10.133.0.180:15021: connect: connection refused" start-of-body=
Oct 30 22:29:28.266768 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:28.266330    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q" podUID="c4886da1-54b1-4252-bb65-553e0ec54d7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.180:15021/healthz/ready\": dial tcp 10.133.0.180:15021: connect: connection refused"
Oct 30 22:29:28.318313 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:28.318205    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"
Oct 30 22:29:28.318650 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:28.318621    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.181:15021/healthz/ready\": dial tcp 10.133.0.181:15021: connect: connection refused" start-of-body=
Oct 30 22:29:28.318836 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:28.318807    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2" podUID="0a69cc06-f8e1-499a-828d-9876d05ed442" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.181:15021/healthz/ready\": dial tcp 10.133.0.181:15021: connect: connection refused"
Oct 30 22:29:29.101820 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:29.100278    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.182:15021/healthz/ready\": dial tcp 10.133.0.182:15021: connect: connection refused" start-of-body=
Oct 30 22:29:29.101820 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:29.101362    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds" podUID="eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.182:15021/healthz/ready\": dial tcp 10.133.0.182:15021: connect: connection refused"
Oct 30 22:29:29.396336 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:29.391032    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.181:15021/healthz/ready\": dial tcp 10.133.0.181:15021: connect: connection refused" start-of-body=
Oct 30 22:29:29.396336 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:29.391097    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2" podUID="0a69cc06-f8e1-499a-828d-9876d05ed442" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.181:15021/healthz/ready\": dial tcp 10.133.0.181:15021: connect: connection refused"
Oct 30 22:29:29.440272 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:29.427426    4871 patch_prober.go:29] interesting pod/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.180:15021/healthz/ready\": dial tcp 10.133.0.180:15021: connect: connection refused" start-of-body=
Oct 30 22:29:29.440272 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:29.427486    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q" podUID="c4886da1-54b1-4252-bb65-553e0ec54d7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.180:15021/healthz/ready\": dial tcp 10.133.0.180:15021: connect: connection refused"
Oct 30 22:29:29.898300 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:29.898253    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76"]
Oct 30 22:29:29.910575 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:29.910538    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76"
Oct 30 22:29:29.947261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:29.944082    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76\" (UID: \"f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76"
Oct 30 22:29:29.947261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:29.944275    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76\" (UID: \"f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76"
Oct 30 22:29:29.947261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:29.944448    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76\" (UID: \"f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76"
Oct 30 22:29:29.947261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:29.944594    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8l6v\" (UniqueName: \"kubernetes.io/projected/f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3-kube-api-access-n8l6v\") pod \"maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76\" (UID: \"f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76"
Oct 30 22:29:30.093972 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:30.040783    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-5e884c-serving-cert\""
Oct 30 22:29:30.356591 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:30.109036    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76\" (UID: \"f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76"
Oct 30 22:29:30.494217 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:30.494175    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8l6v\" (UniqueName: \"kubernetes.io/projected/f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3-kube-api-access-n8l6v\") pod \"maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76\" (UID: \"f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76"
Oct 30 22:29:30.494622 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:30.494593    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76\" (UID: \"f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76"
Oct 30 22:29:30.494871 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:30.494849    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76\" (UID: \"f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76"
Oct 30 22:29:30.495119 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:30.254146    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:30.495344 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:30.110217    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76\" (UID: \"f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76"
Oct 30 22:29:30.495781 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:30.127502    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.182:15021/healthz/ready\": dial tcp 10.133.0.182:15021: connect: connection refused" start-of-body=
Oct 30 22:29:30.495946 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:30.495921    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds" podUID="eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.182:15021/healthz/ready\": dial tcp 10.133.0.182:15021: connect: connection refused"
Oct 30 22:29:30.512441 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:30.249684    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76"]
Oct 30 22:29:30.523929 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:30.278098    4871 patch_prober.go:29] interesting pod/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.180:15021/healthz/ready\": dial tcp 10.133.0.180:15021: connect: connection refused" start-of-body=
Oct 30 22:29:30.524303 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:30.524204    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q" podUID="c4886da1-54b1-4252-bb65-553e0ec54d7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.180:15021/healthz/ready\": dial tcp 10.133.0.180:15021: connect: connection refused"
Oct 30 22:29:30.524497 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:30.320685    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.181:15021/healthz/ready\": dial tcp 10.133.0.181:15021: connect: connection refused" start-of-body=
Oct 30 22:29:30.524742 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:30.524720    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2" podUID="0a69cc06-f8e1-499a-828d-9876d05ed442" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.181:15021/healthz/ready\": dial tcp 10.133.0.181:15021: connect: connection refused"
Oct 30 22:29:30.599387 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:30.599295    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76\" (UID: \"f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76"
Oct 30 22:29:30.632267 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:30.625996    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76\" (UID: \"f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76"
Oct 30 22:29:30.696261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:30.692207    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:30.799681 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:30.799645    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8l6v\" (UniqueName: \"kubernetes.io/projected/f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3-kube-api-access-n8l6v\") pod \"maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76\" (UID: \"f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76"
Oct 30 22:29:31.082993 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:31.082954    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.182:15021/healthz/ready\": dial tcp 10.133.0.182:15021: connect: connection refused" start-of-body=
Oct 30 22:29:31.083353 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:31.083303    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds" podUID="eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.182:15021/healthz/ready\": dial tcp 10.133.0.182:15021: connect: connection refused"
Oct 30 22:29:31.083737 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:31.083716    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76"
Oct 30 22:29:31.263163 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:31.262714    4871 patch_prober.go:29] interesting pod/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.180:15021/healthz/ready\": dial tcp 10.133.0.180:15021: connect: connection refused" start-of-body=
Oct 30 22:29:31.263163 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:31.262781    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q" podUID="c4886da1-54b1-4252-bb65-553e0ec54d7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.180:15021/healthz/ready\": dial tcp 10.133.0.180:15021: connect: connection refused"
Oct 30 22:29:31.316809 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:31.316773    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.181:15021/healthz/ready\": dial tcp 10.133.0.181:15021: connect: connection refused" start-of-body=
Oct 30 22:29:31.317177 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:31.317132    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2" podUID="0a69cc06-f8e1-499a-828d-9876d05ed442" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.181:15021/healthz/ready\": dial tcp 10.133.0.181:15021: connect: connection refused"
Oct 30 22:29:32.152983 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:32.152944    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.182:15021/healthz/ready\": dial tcp 10.133.0.182:15021: connect: connection refused" start-of-body=
Oct 30 22:29:32.153613 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:32.153579    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds" podUID="eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.182:15021/healthz/ready\": dial tcp 10.133.0.182:15021: connect: connection refused"
Oct 30 22:29:32.192860 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:32.192813    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76"]
Oct 30 22:29:32.270942 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:32.266404    4871 patch_prober.go:29] interesting pod/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.180:15021/healthz/ready\": dial tcp 10.133.0.180:15021: connect: connection refused" start-of-body=
Oct 30 22:29:32.270942 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:32.266483    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q" podUID="c4886da1-54b1-4252-bb65-553e0ec54d7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.180:15021/healthz/ready\": dial tcp 10.133.0.180:15021: connect: connection refused"
Oct 30 22:29:32.282328 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:32.280058    4871 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:29:32.332586 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:32.332544    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.181:15021/healthz/ready\": dial tcp 10.133.0.181:15021: connect: connection refused" start-of-body=
Oct 30 22:29:32.332903 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:32.332828    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2" podUID="0a69cc06-f8e1-499a-828d-9876d05ed442" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.181:15021/healthz/ready\": dial tcp 10.133.0.181:15021: connect: connection refused"
Oct 30 22:29:33.001708 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:33.000700    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76" event={"ID":"f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3","Type":"ContainerStarted","Data":"ac13a7d74b1c5618bc409a8a137e2e51cb5d0b0e01eaeee22170680274d9d806"}
Oct 30 22:29:33.088719 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:33.088679    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.182:15021/healthz/ready\": dial tcp 10.133.0.182:15021: connect: connection refused" start-of-body=
Oct 30 22:29:33.089012 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:33.088981    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds" podUID="eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.182:15021/healthz/ready\": dial tcp 10.133.0.182:15021: connect: connection refused"
Oct 30 22:29:33.269550 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:33.269433    4871 patch_prober.go:29] interesting pod/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.180:15021/healthz/ready\": dial tcp 10.133.0.180:15021: connect: connection refused" start-of-body=
Oct 30 22:29:33.270176 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:33.270133    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q" podUID="c4886da1-54b1-4252-bb65-553e0ec54d7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.180:15021/healthz/ready\": dial tcp 10.133.0.180:15021: connect: connection refused"
Oct 30 22:29:33.320586 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:33.320547    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.181:15021/healthz/ready\": dial tcp 10.133.0.181:15021: connect: connection refused" start-of-body=
Oct 30 22:29:33.320855 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:33.320823    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2" podUID="0a69cc06-f8e1-499a-828d-9876d05ed442" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.181:15021/healthz/ready\": dial tcp 10.133.0.181:15021: connect: connection refused"
Oct 30 22:29:34.092800 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:34.092702    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.182:15021/healthz/ready\": dial tcp 10.133.0.182:15021: connect: connection refused" start-of-body=
Oct 30 22:29:34.093061 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:34.093032    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds" podUID="eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.182:15021/healthz/ready\": dial tcp 10.133.0.182:15021: connect: connection refused"
Oct 30 22:29:34.255075 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:34.255029    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76" event={"ID":"f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3","Type":"ContainerStarted","Data":"8df201acc09b5f6df9292f6f520f92cc80134f581f7a22d04675c9a0f0767742"}
Oct 30 22:29:34.256272 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:34.256249    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76"
Oct 30 22:29:34.315623 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:34.315584    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.181:15021/healthz/ready\": dial tcp 10.133.0.181:15021: connect: connection refused" start-of-body=
Oct 30 22:29:34.316260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:34.316201    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2" podUID="0a69cc06-f8e1-499a-828d-9876d05ed442" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.181:15021/healthz/ready\": dial tcp 10.133.0.181:15021: connect: connection refused"
Oct 30 22:29:34.400096 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:34.400063    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"
Oct 30 22:29:34.400987 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:34.400965    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"
Oct 30 22:29:34.423263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:34.411750    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"
Oct 30 22:29:34.608262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:34.600461    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76" podStartSLOduration=4.395213002 podStartE2EDuration="5.600434878s" totalImagesPullingTime="1.205221876s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:29:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:29:34.441544542 +0000 UTC m=+3116.619033160" watchObservedRunningTime="2026-10-30 22:29:34.600434878 +0000 UTC m=+3116.777923495"
Oct 30 22:29:35.082558 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:35.082514    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.182:15021/healthz/ready\": dial tcp 10.133.0.182:15021: connect: connection refused" start-of-body=
Oct 30 22:29:35.082760 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:35.082587    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds" podUID="eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.182:15021/healthz/ready\": dial tcp 10.133.0.182:15021: connect: connection refused"
Oct 30 22:29:35.315975 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:35.315929    4871 patch_prober.go:29] interesting pod/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.181:15021/healthz/ready\": dial tcp 10.133.0.181:15021: connect: connection refused" start-of-body=
Oct 30 22:29:35.316427 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:35.316002    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2" podUID="0a69cc06-f8e1-499a-828d-9876d05ed442" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.181:15021/healthz/ready\": dial tcp 10.133.0.181:15021: connect: connection refused"
Oct 30 22:29:35.569699 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:29:35.569656    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:29:35.969611 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:35.969569    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:36.483289 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:36.483214    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"
Oct 30 22:29:36.483788 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:36.483726    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"
Oct 30 22:29:36.484514 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:36.484494    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"
Oct 30 22:29:36.536973 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:36.536938    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"
Oct 30 22:29:36.537612 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:36.537576    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"
Oct 30 22:29:36.538467 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:36.538437    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"
Oct 30 22:29:40.291496 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:40.291468    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76"
Oct 30 22:29:40.643501 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:40.643455    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:44.885372 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:44.885329    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6"]
Oct 30 22:29:44.890896 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:44.890873    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6"
Oct 30 22:29:44.945488 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:44.945450    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6"]
Oct 30 22:29:44.949569 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:44.949535    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9px97\" (UniqueName: \"kubernetes.io/projected/b461178b-38cb-446a-8b6c-3a615c77a6f5-kube-api-access-9px97\") pod \"maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6\" (UID: \"b461178b-38cb-446a-8b6c-3a615c77a6f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6"
Oct 30 22:29:44.949751 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:44.949583    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b461178b-38cb-446a-8b6c-3a615c77a6f5-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6\" (UID: \"b461178b-38cb-446a-8b6c-3a615c77a6f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6"
Oct 30 22:29:44.949827 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:44.949741    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b461178b-38cb-446a-8b6c-3a615c77a6f5-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6\" (UID: \"b461178b-38cb-446a-8b6c-3a615c77a6f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6"
Oct 30 22:29:44.949918 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:44.949897    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b461178b-38cb-446a-8b6c-3a615c77a6f5-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6\" (UID: \"b461178b-38cb-446a-8b6c-3a615c77a6f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6"
Oct 30 22:29:44.992455 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:44.992422    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-43d592-serving-cert\""
Oct 30 22:29:45.051370 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:45.051330    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9px97\" (UniqueName: \"kubernetes.io/projected/b461178b-38cb-446a-8b6c-3a615c77a6f5-kube-api-access-9px97\") pod \"maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6\" (UID: \"b461178b-38cb-446a-8b6c-3a615c77a6f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6"
Oct 30 22:29:45.051370 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:45.051370    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b461178b-38cb-446a-8b6c-3a615c77a6f5-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6\" (UID: \"b461178b-38cb-446a-8b6c-3a615c77a6f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6"
Oct 30 22:29:45.051645 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:45.051419    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b461178b-38cb-446a-8b6c-3a615c77a6f5-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6\" (UID: \"b461178b-38cb-446a-8b6c-3a615c77a6f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6"
Oct 30 22:29:45.051645 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:45.051495    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b461178b-38cb-446a-8b6c-3a615c77a6f5-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6\" (UID: \"b461178b-38cb-446a-8b6c-3a615c77a6f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6"
Oct 30 22:29:45.051645 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:29:45.051596    4871 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-43d592-serving-cert: secret "maas-api-e2e-shared-a-w3-43d592-serving-cert" not found
Oct 30 22:29:45.051824 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:29:45.051672    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b461178b-38cb-446a-8b6c-3a615c77a6f5-maas-api-tls podName:b461178b-38cb-446a-8b6c-3a615c77a6f5 nodeName:}" failed. No retries permitted until 2026-10-30 22:29:45.55163978 +0000 UTC m=+3127.729128392 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b461178b-38cb-446a-8b6c-3a615c77a6f5-maas-api-tls") pod "maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6" (UID: "b461178b-38cb-446a-8b6c-3a615c77a6f5") : secret "maas-api-e2e-shared-a-w3-43d592-serving-cert" not found
Oct 30 22:29:45.052328 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:45.052302    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b461178b-38cb-446a-8b6c-3a615c77a6f5-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6\" (UID: \"b461178b-38cb-446a-8b6c-3a615c77a6f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6"
Oct 30 22:29:45.054008 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:45.053977    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b461178b-38cb-446a-8b6c-3a615c77a6f5-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6\" (UID: \"b461178b-38cb-446a-8b6c-3a615c77a6f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6"
Oct 30 22:29:45.083575 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:45.083535    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:45.115208 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:45.115166    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9px97\" (UniqueName: \"kubernetes.io/projected/b461178b-38cb-446a-8b6c-3a615c77a6f5-kube-api-access-9px97\") pod \"maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6\" (UID: \"b461178b-38cb-446a-8b6c-3a615c77a6f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6"
Oct 30 22:29:45.238696 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:45.238597    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:45.396814 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:45.396771    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:45.570312 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:45.570207    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b461178b-38cb-446a-8b6c-3a615c77a6f5-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6\" (UID: \"b461178b-38cb-446a-8b6c-3a615c77a6f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6"
Oct 30 22:29:45.572692 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:45.572667    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b461178b-38cb-446a-8b6c-3a615c77a6f5-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6\" (UID: \"b461178b-38cb-446a-8b6c-3a615c77a6f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6"
Oct 30 22:29:45.806408 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:45.806373    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6"
Oct 30 22:29:46.323052 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:29:46.323007    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb461178b_38cb_446a_8b6c_3a615c77a6f5.slice/crio-8086f82244f667d89719780092571d19d52d7f2c026e451739fc622b356f5182 WatchSource:0}: Error finding container 8086f82244f667d89719780092571d19d52d7f2c026e451739fc622b356f5182: Status 404 returned error can't find the container with id 8086f82244f667d89719780092571d19d52d7f2c026e451739fc622b356f5182
Oct 30 22:29:46.327729 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:46.327704    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6"]
Oct 30 22:29:46.484334 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:46.484284    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6" event={"ID":"b461178b-38cb-446a-8b6c-3a615c77a6f5","Type":"ContainerStarted","Data":"8086f82244f667d89719780092571d19d52d7f2c026e451739fc622b356f5182"}
Oct 30 22:29:46.515203 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:46.515159    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:47.495144 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:47.495101    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6" event={"ID":"b461178b-38cb-446a-8b6c-3a615c77a6f5","Type":"ContainerStarted","Data":"16c39c700557679bf8574fb9d7e2b1ac2c19883f0f18e5bd28c24509566599dc"}
Oct 30 22:29:47.495664 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:47.495476    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6"
Oct 30 22:29:47.621773 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:47.621664    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6" podStartSLOduration=2.5759997070000002 podStartE2EDuration="3.621648886s" totalImagesPullingTime="1.045649179s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:29:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:29:47.61807972 +0000 UTC m=+3129.795568338" watchObservedRunningTime="2026-10-30 22:29:47.621648886 +0000 UTC m=+3129.799137502"
Oct 30 22:29:47.975446 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:47.975356    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:48.131261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:48.130871    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:50.346611 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:50.345001    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:53.527148 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:53.527116    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6"
Oct 30 22:29:53.750706 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:53.750670    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:53.914224 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:53.914170    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:54.090605 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:54.090565    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"]
Oct 30 22:29:54.090872 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:54.090835    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q" podUID="c4886da1-54b1-4252-bb65-553e0ec54d7a" containerName="istio-proxy" containerID="cri-o://eb80ca2751b43e6654c61c4fd592440957b2f768eea10ab1dd01bbb8fce304ee" gracePeriod=30
Oct 30 22:29:54.116835 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:54.116796    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:55.020167 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:55.020126    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:57.601730 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:57.601625    4871 generic.go:356] "Generic (PLEG): container finished" podID="ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea" containerID="b5e0536939bf1528ad23a40a01e4190d2a8a8b88d444591654dd83060b68595a" exitCode=0
Oct 30 22:29:57.601730 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:57.601694    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x" event={"ID":"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea","Type":"ContainerDied","Data":"b5e0536939bf1528ad23a40a01e4190d2a8a8b88d444591654dd83060b68595a"}
Oct 30 22:29:57.602314 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:57.602105    4871 scope.go:122] "RemoveContainer" containerID="b5e0536939bf1528ad23a40a01e4190d2a8a8b88d444591654dd83060b68595a"
Oct 30 22:29:57.604038 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:57.604004    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:29:57.604123 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:57.604106    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:29:57.604171 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:57.604156    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:29:58.636176 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:58.636140    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x" event={"ID":"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea","Type":"ContainerStarted","Data":"a7d1b0220ae04364eacc4053d39b06b2ef39977da2a77ac4708ec91e40e2a431"}
Oct 30 22:29:59.135972 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.135946    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"
Oct 30 22:29:59.136684 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.136659    4871 patch_prober.go:29] interesting pod/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.172:15021/healthz/ready\": dial tcp 10.133.0.172:15021: connect: connection refused" start-of-body=
Oct 30 22:29:59.136783 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.136710    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x" podUID="ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.172:15021/healthz/ready\": dial tcp 10.133.0.172:15021: connect: connection refused"
Oct 30 22:29:59.300363 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.300329    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj"]
Oct 30 22:29:59.337946 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.337913    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj"
Oct 30 22:29:59.363169 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.363133    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj"]
Oct 30 22:29:59.374436 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.374013    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2cc9fd6-d809-460e-a2dc-dc6ea02d3592-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj\" (UID: \"f2cc9fd6-d809-460e-a2dc-dc6ea02d3592\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj"
Oct 30 22:29:59.374436 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.374149    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89hxn\" (UniqueName: \"kubernetes.io/projected/f2cc9fd6-d809-460e-a2dc-dc6ea02d3592-kube-api-access-89hxn\") pod \"maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj\" (UID: \"f2cc9fd6-d809-460e-a2dc-dc6ea02d3592\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj"
Oct 30 22:29:59.374436 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.374181    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2cc9fd6-d809-460e-a2dc-dc6ea02d3592-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj\" (UID: \"f2cc9fd6-d809-460e-a2dc-dc6ea02d3592\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj"
Oct 30 22:29:59.374436 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.374255    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2cc9fd6-d809-460e-a2dc-dc6ea02d3592-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj\" (UID: \"f2cc9fd6-d809-460e-a2dc-dc6ea02d3592\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj"
Oct 30 22:29:59.393336 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.393216    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w0-6fc60c-serving-cert\""
Oct 30 22:29:59.457559 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.457477    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"
Oct 30 22:29:59.484099 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.483021    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89hxn\" (UniqueName: \"kubernetes.io/projected/f2cc9fd6-d809-460e-a2dc-dc6ea02d3592-kube-api-access-89hxn\") pod \"maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj\" (UID: \"f2cc9fd6-d809-460e-a2dc-dc6ea02d3592\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj"
Oct 30 22:29:59.484099 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.483107    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2cc9fd6-d809-460e-a2dc-dc6ea02d3592-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj\" (UID: \"f2cc9fd6-d809-460e-a2dc-dc6ea02d3592\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj"
Oct 30 22:29:59.484099 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.483198    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2cc9fd6-d809-460e-a2dc-dc6ea02d3592-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj\" (UID: \"f2cc9fd6-d809-460e-a2dc-dc6ea02d3592\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj"
Oct 30 22:29:59.484099 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.483536    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2cc9fd6-d809-460e-a2dc-dc6ea02d3592-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj\" (UID: \"f2cc9fd6-d809-460e-a2dc-dc6ea02d3592\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj"
Oct 30 22:29:59.484099 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:29:59.483730    4871 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-6fc60c-serving-cert: secret "maas-api-e2e-shared-a-w0-6fc60c-serving-cert" not found
Oct 30 22:29:59.484099 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:29:59.483820    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2cc9fd6-d809-460e-a2dc-dc6ea02d3592-maas-api-tls podName:f2cc9fd6-d809-460e-a2dc-dc6ea02d3592 nodeName:}" failed. No retries permitted until 2026-10-30 22:29:59.983798526 +0000 UTC m=+3142.161287134 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f2cc9fd6-d809-460e-a2dc-dc6ea02d3592-maas-api-tls") pod "maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj" (UID: "f2cc9fd6-d809-460e-a2dc-dc6ea02d3592") : secret "maas-api-e2e-shared-a-w0-6fc60c-serving-cert" not found
Oct 30 22:29:59.484518 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.484190    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2cc9fd6-d809-460e-a2dc-dc6ea02d3592-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj\" (UID: \"f2cc9fd6-d809-460e-a2dc-dc6ea02d3592\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj"
Oct 30 22:29:59.486798 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.486769    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2cc9fd6-d809-460e-a2dc-dc6ea02d3592-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj\" (UID: \"f2cc9fd6-d809-460e-a2dc-dc6ea02d3592\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj"
Oct 30 22:29:59.534551 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.534499    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:59.643263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.591652    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4886da1-54b1-4252-bb65-553e0ec54d7a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4886da1-54b1-4252-bb65-553e0ec54d7a-istio-data\") pod \"c4886da1-54b1-4252-bb65-553e0ec54d7a\" (UID: \"c4886da1-54b1-4252-bb65-553e0ec54d7a\") "
Oct 30 22:29:59.643263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.591755    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c4886da1-54b1-4252-bb65-553e0ec54d7a-kube-api-access-jg65z\" (UniqueName: \"kubernetes.io/projected/c4886da1-54b1-4252-bb65-553e0ec54d7a-kube-api-access-jg65z\") pod \"c4886da1-54b1-4252-bb65-553e0ec54d7a\" (UID: \"c4886da1-54b1-4252-bb65-553e0ec54d7a\") "
Oct 30 22:29:59.643263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.591873    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c4886da1-54b1-4252-bb65-553e0ec54d7a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4886da1-54b1-4252-bb65-553e0ec54d7a-istio-podinfo\") pod \"c4886da1-54b1-4252-bb65-553e0ec54d7a\" (UID: \"c4886da1-54b1-4252-bb65-553e0ec54d7a\") "
Oct 30 22:29:59.643263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.591918    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4886da1-54b1-4252-bb65-553e0ec54d7a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4886da1-54b1-4252-bb65-553e0ec54d7a-credential-socket\") pod \"c4886da1-54b1-4252-bb65-553e0ec54d7a\" (UID: \"c4886da1-54b1-4252-bb65-553e0ec54d7a\") "
Oct 30 22:29:59.643263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.591946    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4886da1-54b1-4252-bb65-553e0ec54d7a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4886da1-54b1-4252-bb65-553e0ec54d7a-workload-socket\") pod \"c4886da1-54b1-4252-bb65-553e0ec54d7a\" (UID: \"c4886da1-54b1-4252-bb65-553e0ec54d7a\") "
Oct 30 22:29:59.643263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.592032    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4886da1-54b1-4252-bb65-553e0ec54d7a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4886da1-54b1-4252-bb65-553e0ec54d7a-workload-certs\") pod \"c4886da1-54b1-4252-bb65-553e0ec54d7a\" (UID: \"c4886da1-54b1-4252-bb65-553e0ec54d7a\") "
Oct 30 22:29:59.643263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.592073    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c4886da1-54b1-4252-bb65-553e0ec54d7a-istio-token\" (UniqueName: \"kubernetes.io/projected/c4886da1-54b1-4252-bb65-553e0ec54d7a-istio-token\") pod \"c4886da1-54b1-4252-bb65-553e0ec54d7a\" (UID: \"c4886da1-54b1-4252-bb65-553e0ec54d7a\") "
Oct 30 22:29:59.643263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.592103    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c4886da1-54b1-4252-bb65-553e0ec54d7a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4886da1-54b1-4252-bb65-553e0ec54d7a-istiod-ca-cert\") pod \"c4886da1-54b1-4252-bb65-553e0ec54d7a\" (UID: \"c4886da1-54b1-4252-bb65-553e0ec54d7a\") "
Oct 30 22:29:59.643263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.592147    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4886da1-54b1-4252-bb65-553e0ec54d7a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4886da1-54b1-4252-bb65-553e0ec54d7a-istio-envoy\") pod \"c4886da1-54b1-4252-bb65-553e0ec54d7a\" (UID: \"c4886da1-54b1-4252-bb65-553e0ec54d7a\") "
Oct 30 22:29:59.643263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.606353    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4886da1-54b1-4252-bb65-553e0ec54d7a-workload-certs" pod "c4886da1-54b1-4252-bb65-553e0ec54d7a" (UID: "c4886da1-54b1-4252-bb65-553e0ec54d7a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:59.643263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.606650    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4886da1-54b1-4252-bb65-553e0ec54d7a-workload-socket" pod "c4886da1-54b1-4252-bb65-553e0ec54d7a" (UID: "c4886da1-54b1-4252-bb65-553e0ec54d7a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:59.643263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.607393    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4886da1-54b1-4252-bb65-553e0ec54d7a-istiod-ca-cert" pod "c4886da1-54b1-4252-bb65-553e0ec54d7a" (UID: "c4886da1-54b1-4252-bb65-553e0ec54d7a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:29:59.643263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.621429    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4886da1-54b1-4252-bb65-553e0ec54d7a-credential-socket" pod "c4886da1-54b1-4252-bb65-553e0ec54d7a" (UID: "c4886da1-54b1-4252-bb65-553e0ec54d7a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:59.643263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.621866    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4886da1-54b1-4252-bb65-553e0ec54d7a-istio-data" pod "c4886da1-54b1-4252-bb65-553e0ec54d7a" (UID: "c4886da1-54b1-4252-bb65-553e0ec54d7a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:59.651550 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.651462    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4886da1-54b1-4252-bb65-553e0ec54d7a-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:29:59.651550 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.651496    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4886da1-54b1-4252-bb65-553e0ec54d7a-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:29:59.651550 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.651509    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4886da1-54b1-4252-bb65-553e0ec54d7a-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:29:59.651550 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.651521    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4886da1-54b1-4252-bb65-553e0ec54d7a-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:29:59.651550 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.651536    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4886da1-54b1-4252-bb65-553e0ec54d7a-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:29:59.663449 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.663411    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4886da1-54b1-4252-bb65-553e0ec54d7a-istio-token" pod "c4886da1-54b1-4252-bb65-553e0ec54d7a" (UID: "c4886da1-54b1-4252-bb65-553e0ec54d7a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:29:59.664591 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.664401    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-89hxn\" (UniqueName: \"kubernetes.io/projected/f2cc9fd6-d809-460e-a2dc-dc6ea02d3592-kube-api-access-89hxn\") pod \"maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj\" (UID: \"f2cc9fd6-d809-460e-a2dc-dc6ea02d3592\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj"
Oct 30 22:29:59.664738 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.664710    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c4886da1-54b1-4252-bb65-553e0ec54d7a-istio-podinfo" pod "c4886da1-54b1-4252-bb65-553e0ec54d7a" (UID: "c4886da1-54b1-4252-bb65-553e0ec54d7a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:29:59.666090 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.666057    4871 generic.go:356] "Generic (PLEG): container finished" podID="c4886da1-54b1-4252-bb65-553e0ec54d7a" containerID="eb80ca2751b43e6654c61c4fd592440957b2f768eea10ab1dd01bbb8fce304ee" exitCode=0
Oct 30 22:29:59.666286 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.666252    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q" event={"ID":"c4886da1-54b1-4252-bb65-553e0ec54d7a","Type":"ContainerDied","Data":"eb80ca2751b43e6654c61c4fd592440957b2f768eea10ab1dd01bbb8fce304ee"}
Oct 30 22:29:59.667214 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.667187    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q" event={"ID":"c4886da1-54b1-4252-bb65-553e0ec54d7a","Type":"ContainerDied","Data":"cf09da268cdbbc118abcf5ff537be9c6a90972828b54ce64c29aa4750208f621"}
Oct 30 22:29:59.667315 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.667284    4871 scope.go:122] "RemoveContainer" containerID="eb80ca2751b43e6654c61c4fd592440957b2f768eea10ab1dd01bbb8fce304ee"
Oct 30 22:29:59.667419 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.667036    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"
Oct 30 22:29:59.668181 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.668156    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4886da1-54b1-4252-bb65-553e0ec54d7a-istio-envoy" pod "c4886da1-54b1-4252-bb65-553e0ec54d7a" (UID: "c4886da1-54b1-4252-bb65-553e0ec54d7a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:59.668615 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.668593    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4886da1-54b1-4252-bb65-553e0ec54d7a-kube-api-access-jg65z" pod "c4886da1-54b1-4252-bb65-553e0ec54d7a" (UID: "c4886da1-54b1-4252-bb65-553e0ec54d7a"). InnerVolumeSpecName "kube-api-access-jg65z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:29:59.683467 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.683330    4871 scope.go:122] "RemoveContainer" containerID="eb80ca2751b43e6654c61c4fd592440957b2f768eea10ab1dd01bbb8fce304ee"
Oct 30 22:29:59.683869 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:29:59.683782    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb80ca2751b43e6654c61c4fd592440957b2f768eea10ab1dd01bbb8fce304ee\": container with ID starting with eb80ca2751b43e6654c61c4fd592440957b2f768eea10ab1dd01bbb8fce304ee not found: ID does not exist" containerID="eb80ca2751b43e6654c61c4fd592440957b2f768eea10ab1dd01bbb8fce304ee"
Oct 30 22:29:59.683869 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.683821    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb80ca2751b43e6654c61c4fd592440957b2f768eea10ab1dd01bbb8fce304ee"} err="failed to get container status \"eb80ca2751b43e6654c61c4fd592440957b2f768eea10ab1dd01bbb8fce304ee\": rpc error: code = NotFound desc = could not find container \"eb80ca2751b43e6654c61c4fd592440957b2f768eea10ab1dd01bbb8fce304ee\": container with ID starting with eb80ca2751b43e6654c61c4fd592440957b2f768eea10ab1dd01bbb8fce304ee not found: ID does not exist"
Oct 30 22:29:59.728323 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.712849    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:59.795259 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.753468    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jg65z\" (UniqueName: \"kubernetes.io/projected/c4886da1-54b1-4252-bb65-553e0ec54d7a-kube-api-access-jg65z\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:29:59.795592 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.795533    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4886da1-54b1-4252-bb65-553e0ec54d7a-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:29:59.795745 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.795732    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4886da1-54b1-4252-bb65-553e0ec54d7a-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:29:59.795851 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.795840    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4886da1-54b1-4252-bb65-553e0ec54d7a-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:29:59.864888 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:29:59.864848    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:00.034495 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:00.034451    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2cc9fd6-d809-460e-a2dc-dc6ea02d3592-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj\" (UID: \"f2cc9fd6-d809-460e-a2dc-dc6ea02d3592\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj"
Oct 30 22:30:00.039555 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:00.039521    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2cc9fd6-d809-460e-a2dc-dc6ea02d3592-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj\" (UID: \"f2cc9fd6-d809-460e-a2dc-dc6ea02d3592\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj"
Oct 30 22:30:00.114263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:00.103184    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"]
Oct 30 22:30:00.127260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:00.126467    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-a-f7c488-openshift-default-54468fb977-dj64q"]
Oct 30 22:30:00.136742 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:00.136706    4871 patch_prober.go:29] interesting pod/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.172:15021/healthz/ready\": dial tcp 10.133.0.172:15021: connect: connection refused" start-of-body=
Oct 30 22:30:00.136933 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:00.136775    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x" podUID="ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.172:15021/healthz/ready\": dial tcp 10.133.0.172:15021: connect: connection refused"
Oct 30 22:30:00.258723 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:00.258624    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj"
Oct 30 22:30:00.735027 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:00.734997    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj"]
Oct 30 22:30:01.136856 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:01.136820    4871 patch_prober.go:29] interesting pod/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.172:15021/healthz/ready\": dial tcp 10.133.0.172:15021: connect: connection refused" start-of-body=
Oct 30 22:30:01.137052 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:01.136894    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x" podUID="ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.172:15021/healthz/ready\": dial tcp 10.133.0.172:15021: connect: connection refused"
Oct 30 22:30:01.686623 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:01.686588    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj" event={"ID":"f2cc9fd6-d809-460e-a2dc-dc6ea02d3592","Type":"ContainerStarted","Data":"00162065a904e8d6c654f222579946e94ff603415be43e9767b9f59bab80c05e"}
Oct 30 22:30:01.962184 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:01.962101    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c4886da1-54b1-4252-bb65-553e0ec54d7a" path="/var/lib/kubelet/pods/c4886da1-54b1-4252-bb65-553e0ec54d7a/volumes"
Oct 30 22:30:02.136630 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:02.136591    4871 patch_prober.go:29] interesting pod/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.172:15021/healthz/ready\": dial tcp 10.133.0.172:15021: connect: connection refused" start-of-body=
Oct 30 22:30:02.136811 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:02.136663    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x" podUID="ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.172:15021/healthz/ready\": dial tcp 10.133.0.172:15021: connect: connection refused"
Oct 30 22:30:02.696935 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:02.696892    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj" event={"ID":"f2cc9fd6-d809-460e-a2dc-dc6ea02d3592","Type":"ContainerStarted","Data":"13bb5f870006cbc04f4b776b20d76b208a5e36b609358580955ce416c2d321e7"}
Oct 30 22:30:02.697180 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:02.697025    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj"
Oct 30 22:30:02.833094 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:02.832872    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj" podStartSLOduration=2.885864571 podStartE2EDuration="3.832850454s" totalImagesPullingTime="946.985883ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:29:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:30:02.830682947 +0000 UTC m=+3145.008171565" watchObservedRunningTime="2026-10-30 22:30:02.832850454 +0000 UTC m=+3145.010339076"
Oct 30 22:30:03.136200 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:03.136155    4871 patch_prober.go:29] interesting pod/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.172:15021/healthz/ready\": dial tcp 10.133.0.172:15021: connect: connection refused" start-of-body=
Oct 30 22:30:03.136716 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:03.136230    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x" podUID="ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.172:15021/healthz/ready\": dial tcp 10.133.0.172:15021: connect: connection refused"
Oct 30 22:30:03.732916 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:03.732868    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"
Oct 30 22:30:04.141417 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:04.141377    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"
Oct 30 22:30:04.519871 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:04.519783    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:04.720098 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:04.720068    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"
Oct 30 22:30:05.568570 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:30:05.568533    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:30:08.713953 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:08.713922    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj"
Oct 30 22:30:09.129755 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:09.129712    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:13.622017 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:13.621974    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2"]
Oct 30 22:30:13.622971 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:13.622943    4871 cpu_manager.go:414] "containerMap: removing container" podUID="c4886da1-54b1-4252-bb65-553e0ec54d7a" containerName="istio-proxy"
Oct 30 22:30:13.622971 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:13.622972    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c4886da1-54b1-4252-bb65-553e0ec54d7a" containerName="istio-proxy"
Oct 30 22:30:13.623168 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:13.623155    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="c4886da1-54b1-4252-bb65-553e0ec54d7a" containerName="istio-proxy"
Oct 30 22:30:13.627556 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:13.627531    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2"
Oct 30 22:30:13.685327 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:13.685287    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2"]
Oct 30 22:30:13.734273 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:13.723473    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/533efba5-8b6c-4726-b0eb-2f7b53ac5ea3-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2\" (UID: \"533efba5-8b6c-4726-b0eb-2f7b53ac5ea3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2"
Oct 30 22:30:13.734273 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:13.723544    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xsrk\" (UniqueName: \"kubernetes.io/projected/533efba5-8b6c-4726-b0eb-2f7b53ac5ea3-kube-api-access-2xsrk\") pod \"maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2\" (UID: \"533efba5-8b6c-4726-b0eb-2f7b53ac5ea3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2"
Oct 30 22:30:13.734273 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:13.723604    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/533efba5-8b6c-4726-b0eb-2f7b53ac5ea3-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2\" (UID: \"533efba5-8b6c-4726-b0eb-2f7b53ac5ea3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2"
Oct 30 22:30:13.734273 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:13.723643    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/533efba5-8b6c-4726-b0eb-2f7b53ac5ea3-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2\" (UID: \"533efba5-8b6c-4726-b0eb-2f7b53ac5ea3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2"
Oct 30 22:30:13.734694 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:13.734490    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w1-ef00a4-serving-cert\""
Oct 30 22:30:13.828770 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:13.828721    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/533efba5-8b6c-4726-b0eb-2f7b53ac5ea3-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2\" (UID: \"533efba5-8b6c-4726-b0eb-2f7b53ac5ea3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2"
Oct 30 22:30:13.828999 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:13.828806    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xsrk\" (UniqueName: \"kubernetes.io/projected/533efba5-8b6c-4726-b0eb-2f7b53ac5ea3-kube-api-access-2xsrk\") pod \"maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2\" (UID: \"533efba5-8b6c-4726-b0eb-2f7b53ac5ea3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2"
Oct 30 22:30:13.828999 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:13.828976    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/533efba5-8b6c-4726-b0eb-2f7b53ac5ea3-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2\" (UID: \"533efba5-8b6c-4726-b0eb-2f7b53ac5ea3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2"
Oct 30 22:30:13.829132 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:13.829034    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/533efba5-8b6c-4726-b0eb-2f7b53ac5ea3-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2\" (UID: \"533efba5-8b6c-4726-b0eb-2f7b53ac5ea3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2"
Oct 30 22:30:13.830116 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:13.830089    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/533efba5-8b6c-4726-b0eb-2f7b53ac5ea3-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2\" (UID: \"533efba5-8b6c-4726-b0eb-2f7b53ac5ea3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2"
Oct 30 22:30:13.830483 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:30:13.830450    4871 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ef00a4-serving-cert: secret "maas-api-e2e-shared-a-w1-ef00a4-serving-cert" not found
Oct 30 22:30:13.830721 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:30:13.830705    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/533efba5-8b6c-4726-b0eb-2f7b53ac5ea3-maas-api-tls podName:533efba5-8b6c-4726-b0eb-2f7b53ac5ea3 nodeName:}" failed. No retries permitted until 2026-10-30 22:30:14.33066918 +0000 UTC m=+3156.508157775 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/533efba5-8b6c-4726-b0eb-2f7b53ac5ea3-maas-api-tls") pod "maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2" (UID: "533efba5-8b6c-4726-b0eb-2f7b53ac5ea3") : secret "maas-api-e2e-shared-a-w1-ef00a4-serving-cert" not found
Oct 30 22:30:13.839167 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:13.839135    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/533efba5-8b6c-4726-b0eb-2f7b53ac5ea3-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2\" (UID: \"533efba5-8b6c-4726-b0eb-2f7b53ac5ea3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2"
Oct 30 22:30:13.886687 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:13.886597    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:13.894535 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:13.894506    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xsrk\" (UniqueName: \"kubernetes.io/projected/533efba5-8b6c-4726-b0eb-2f7b53ac5ea3-kube-api-access-2xsrk\") pod \"maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2\" (UID: \"533efba5-8b6c-4726-b0eb-2f7b53ac5ea3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2"
Oct 30 22:30:14.040852 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:14.040815    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:14.216589 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:14.216490    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:14.346154 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:14.346119    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/533efba5-8b6c-4726-b0eb-2f7b53ac5ea3-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2\" (UID: \"533efba5-8b6c-4726-b0eb-2f7b53ac5ea3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2"
Oct 30 22:30:14.348817 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:14.348789    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/533efba5-8b6c-4726-b0eb-2f7b53ac5ea3-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2\" (UID: \"533efba5-8b6c-4726-b0eb-2f7b53ac5ea3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2"
Oct 30 22:30:14.546943 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:14.546838    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2"
Oct 30 22:30:14.941199 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:14.941164    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2"]
Oct 30 22:30:15.815232 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:15.815183    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2" event={"ID":"533efba5-8b6c-4726-b0eb-2f7b53ac5ea3","Type":"ContainerStarted","Data":"8d0175cbed161274e712ae437ef78d6613f6d832faa9eae61113c2444508ce3c"}
Oct 30 22:30:16.827777 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:16.827731    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2" event={"ID":"533efba5-8b6c-4726-b0eb-2f7b53ac5ea3","Type":"ContainerStarted","Data":"6c4e45948ea85a178300e7028108424dfe1a30bcf241ee98312a2892071edaca"}
Oct 30 22:30:16.828386 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:16.827808    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2"
Oct 30 22:30:16.954065 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:16.953992    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2" podStartSLOduration=3.118116417 podStartE2EDuration="3.953970813s" totalImagesPullingTime="835.854396ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:30:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:30:16.950175183 +0000 UTC m=+3159.127663814" watchObservedRunningTime="2026-10-30 22:30:16.953970813 +0000 UTC m=+3159.131459433"
Oct 30 22:30:19.339386 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:19.339340    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:21.493979 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:21.493950    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:21.812056 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:21.811941    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:22.868346 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:22.868312    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2"
Oct 30 22:30:23.148327 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:23.127175    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:25.018859 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:25.018774    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:27.232468 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:27.232411    4871 patch_prober.go:29] interesting pod/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.179:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:30:27.233391 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:27.232497    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2" podUID="478b54f8-6c41-42ad-ac53-cb6144f07773" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.179:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:30:30.002884 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:30.002848    4871 generic.go:356] "Generic (PLEG): container finished" podID="239ef6a2-b9f8-4247-9db4-fc4192f6bf1c" containerID="373bdb1823397318e10f3e052140772b7c13846f314a3bb9840627af7ca2c82a" exitCode=0
Oct 30 22:30:30.003397 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:30.002924    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s" event={"ID":"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c","Type":"ContainerDied","Data":"373bdb1823397318e10f3e052140772b7c13846f314a3bb9840627af7ca2c82a"}
Oct 30 22:30:30.003459 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:30.003405    4871 scope.go:122] "RemoveContainer" containerID="373bdb1823397318e10f3e052140772b7c13846f314a3bb9840627af7ca2c82a"
Oct 30 22:30:30.004994 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:30.004966    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:30:30.005103 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:30.005032    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:30:30.005103 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:30.005066    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:30:30.193860 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:30.193823    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:31.034417 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:31.034376    4871 generic.go:356] "Generic (PLEG): container finished" podID="478b54f8-6c41-42ad-ac53-cb6144f07773" containerID="f6f414d401e6b5799505fff1643916daf0a5f8c0be737f58c1484a5a7b96a6bc" exitCode=0
Oct 30 22:30:31.034906 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:31.034450    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2" event={"ID":"478b54f8-6c41-42ad-ac53-cb6144f07773","Type":"ContainerDied","Data":"f6f414d401e6b5799505fff1643916daf0a5f8c0be737f58c1484a5a7b96a6bc"}
Oct 30 22:30:31.034975 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:31.034956    4871 scope.go:122] "RemoveContainer" containerID="f6f414d401e6b5799505fff1643916daf0a5f8c0be737f58c1484a5a7b96a6bc"
Oct 30 22:30:31.036780 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:31.036747    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:30:31.037027 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:31.037002    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:30:31.037129 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:31.037062    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:30:31.037129 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:31.037074    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s" event={"ID":"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c","Type":"ContainerStarted","Data":"7e7a69db017786d9da7baa4acc750763a8155a6f38da5542554caa52b33c2157"}
Oct 30 22:30:31.414741 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:31.414703    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"
Oct 30 22:30:31.415188 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:31.415035    4871 patch_prober.go:29] interesting pod/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.173:15021/healthz/ready\": dial tcp 10.133.0.173:15021: connect: connection refused" start-of-body=
Oct 30 22:30:31.415403 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:31.415379    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s" podUID="239ef6a2-b9f8-4247-9db4-fc4192f6bf1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.173:15021/healthz/ready\": dial tcp 10.133.0.173:15021: connect: connection refused"
Oct 30 22:30:32.072281 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:32.072231    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2" event={"ID":"478b54f8-6c41-42ad-ac53-cb6144f07773","Type":"ContainerStarted","Data":"e34feda1f78b9316752feaba026dbdb622b99d59b3912538139b14ad5cc00a92"}
Oct 30 22:30:32.180944 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:32.180910    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"
Oct 30 22:30:32.181126 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:32.181103    4871 patch_prober.go:29] interesting pod/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.179:15021/healthz/ready\": dial tcp 10.133.0.179:15021: connect: connection refused" start-of-body=
Oct 30 22:30:32.181212 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:32.181151    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2" podUID="478b54f8-6c41-42ad-ac53-cb6144f07773" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.179:15021/healthz/ready\": dial tcp 10.133.0.179:15021: connect: connection refused"
Oct 30 22:30:32.308263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:32.308209    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:32.406675 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:32.406634    4871 patch_prober.go:29] interesting pod/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.173:15021/healthz/ready\": dial tcp 10.133.0.173:15021: connect: connection refused" start-of-body=
Oct 30 22:30:32.406982 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:32.406951    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s" podUID="239ef6a2-b9f8-4247-9db4-fc4192f6bf1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.173:15021/healthz/ready\": dial tcp 10.133.0.173:15021: connect: connection refused"
Oct 30 22:30:33.180493 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:33.180451    4871 patch_prober.go:29] interesting pod/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.179:15021/healthz/ready\": dial tcp 10.133.0.179:15021: connect: connection refused" start-of-body=
Oct 30 22:30:33.180954 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:33.180524    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2" podUID="478b54f8-6c41-42ad-ac53-cb6144f07773" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.179:15021/healthz/ready\": dial tcp 10.133.0.179:15021: connect: connection refused"
Oct 30 22:30:33.406714 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:33.406665    4871 patch_prober.go:29] interesting pod/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.173:15021/healthz/ready\": dial tcp 10.133.0.173:15021: connect: connection refused" start-of-body=
Oct 30 22:30:33.407074 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:33.407034    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s" podUID="239ef6a2-b9f8-4247-9db4-fc4192f6bf1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.173:15021/healthz/ready\": dial tcp 10.133.0.173:15021: connect: connection refused"
Oct 30 22:30:34.181080 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:34.181041    4871 patch_prober.go:29] interesting pod/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.179:15021/healthz/ready\": dial tcp 10.133.0.179:15021: connect: connection refused" start-of-body=
Oct 30 22:30:34.181523 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:34.181100    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2" podUID="478b54f8-6c41-42ad-ac53-cb6144f07773" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.179:15021/healthz/ready\": dial tcp 10.133.0.179:15021: connect: connection refused"
Oct 30 22:30:34.412272 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:34.411359    4871 patch_prober.go:29] interesting pod/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.173:15021/healthz/ready\": dial tcp 10.133.0.173:15021: connect: connection refused" start-of-body=
Oct 30 22:30:34.412272 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:34.411426    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s" podUID="239ef6a2-b9f8-4247-9db4-fc4192f6bf1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.173:15021/healthz/ready\": dial tcp 10.133.0.173:15021: connect: connection refused"
Oct 30 22:30:35.180617 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:35.180573    4871 patch_prober.go:29] interesting pod/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.179:15021/healthz/ready\": dial tcp 10.133.0.179:15021: connect: connection refused" start-of-body=
Oct 30 22:30:35.180815 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:35.180634    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2" podUID="478b54f8-6c41-42ad-ac53-cb6144f07773" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.179:15021/healthz/ready\": dial tcp 10.133.0.179:15021: connect: connection refused"
Oct 30 22:30:35.406688 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:35.406646    4871 patch_prober.go:29] interesting pod/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.173:15021/healthz/ready\": dial tcp 10.133.0.173:15021: connect: connection refused" start-of-body=
Oct 30 22:30:35.407317 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:35.407280    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s" podUID="239ef6a2-b9f8-4247-9db4-fc4192f6bf1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.173:15021/healthz/ready\": dial tcp 10.133.0.173:15021: connect: connection refused"
Oct 30 22:30:35.593034 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:30:35.592932    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:30:35.667136 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:35.667086    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:36.180494 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:36.180456    4871 patch_prober.go:29] interesting pod/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.179:15021/healthz/ready\": dial tcp 10.133.0.179:15021: connect: connection refused" start-of-body=
Oct 30 22:30:36.180692 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:36.180513    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2" podUID="478b54f8-6c41-42ad-ac53-cb6144f07773" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.179:15021/healthz/ready\": dial tcp 10.133.0.179:15021: connect: connection refused"
Oct 30 22:30:36.429653 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:36.429614    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"
Oct 30 22:30:36.430120 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:36.430096    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"
Oct 30 22:30:36.431078 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:36.431003    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"
Oct 30 22:30:37.180589 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:37.180557    4871 patch_prober.go:29] interesting pod/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.179:15021/healthz/ready\": dial tcp 10.133.0.179:15021: connect: connection refused" start-of-body=
Oct 30 22:30:37.180782 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:37.180610    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2" podUID="478b54f8-6c41-42ad-ac53-cb6144f07773" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.179:15021/healthz/ready\": dial tcp 10.133.0.179:15021: connect: connection refused"
Oct 30 22:30:38.185800 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:38.185768    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"
Oct 30 22:30:38.186285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:38.186233    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"
Oct 30 22:30:38.186951 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:38.186933    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"
Oct 30 22:30:40.844842 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:40.844800    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:42.549579 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:42.549526    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:43.027806 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:43.027766    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:43.692093 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:43.692052    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:43.853398 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:43.853340    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:44.016787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:44.016696    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:45.971870 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:45.971836    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:46.833660 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:46.833567    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:47.503280 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:47.502114    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:47.681918 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:47.680365    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:48.350484 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:48.350403    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:50.363883 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:50.363848    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd"]
Oct 30 22:30:50.364366 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:50.364065    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd" podUID="38c81c78-6dca-4ec6-9347-bf512bcc0d05" containerName="maas-api" containerID="cri-o://c354fae03e416c897090f452786797e3c3b9ad2824eeadd3b7035297acd1b121" gracePeriod=30
Oct 30 22:30:50.489511 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:50.489172    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:50.735272 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:50.735018    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:50.878809 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:50.878763    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd"
Oct 30 22:30:50.985127 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:50.985087    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/38c81c78-6dca-4ec6-9347-bf512bcc0d05-kube-api-access-4m5cr\" (UniqueName: \"kubernetes.io/projected/38c81c78-6dca-4ec6-9347-bf512bcc0d05-kube-api-access-4m5cr\") pod \"38c81c78-6dca-4ec6-9347-bf512bcc0d05\" (UID: \"38c81c78-6dca-4ec6-9347-bf512bcc0d05\") "
Oct 30 22:30:50.985127 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:50.985132    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/38c81c78-6dca-4ec6-9347-bf512bcc0d05-metrics-certs\" (UniqueName: \"kubernetes.io/secret/38c81c78-6dca-4ec6-9347-bf512bcc0d05-metrics-certs\") pod \"38c81c78-6dca-4ec6-9347-bf512bcc0d05\" (UID: \"38c81c78-6dca-4ec6-9347-bf512bcc0d05\") "
Oct 30 22:30:50.985542 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:50.985165    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/38c81c78-6dca-4ec6-9347-bf512bcc0d05-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38c81c78-6dca-4ec6-9347-bf512bcc0d05-openshift-service-ca\") pod \"38c81c78-6dca-4ec6-9347-bf512bcc0d05\" (UID: \"38c81c78-6dca-4ec6-9347-bf512bcc0d05\") "
Oct 30 22:30:50.985542 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:50.985412    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/38c81c78-6dca-4ec6-9347-bf512bcc0d05-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38c81c78-6dca-4ec6-9347-bf512bcc0d05-maas-api-tls\") pod \"38c81c78-6dca-4ec6-9347-bf512bcc0d05\" (UID: \"38c81c78-6dca-4ec6-9347-bf512bcc0d05\") "
Oct 30 22:30:50.985685 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:50.985618    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38c81c78-6dca-4ec6-9347-bf512bcc0d05-openshift-service-ca" pod "38c81c78-6dca-4ec6-9347-bf512bcc0d05" (UID: "38c81c78-6dca-4ec6-9347-bf512bcc0d05"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:30:50.986332 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:50.986180    4871 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38c81c78-6dca-4ec6-9347-bf512bcc0d05-openshift-service-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:30:50.987656 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:50.987606    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38c81c78-6dca-4ec6-9347-bf512bcc0d05-kube-api-access-4m5cr" pod "38c81c78-6dca-4ec6-9347-bf512bcc0d05" (UID: "38c81c78-6dca-4ec6-9347-bf512bcc0d05"). InnerVolumeSpecName "kube-api-access-4m5cr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:30:50.987785 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:50.987677    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38c81c78-6dca-4ec6-9347-bf512bcc0d05-metrics-certs" pod "38c81c78-6dca-4ec6-9347-bf512bcc0d05" (UID: "38c81c78-6dca-4ec6-9347-bf512bcc0d05"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:30:50.987785 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:50.987694    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38c81c78-6dca-4ec6-9347-bf512bcc0d05-maas-api-tls" pod "38c81c78-6dca-4ec6-9347-bf512bcc0d05" (UID: "38c81c78-6dca-4ec6-9347-bf512bcc0d05"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:30:51.087002 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:51.086960    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4m5cr\" (UniqueName: \"kubernetes.io/projected/38c81c78-6dca-4ec6-9347-bf512bcc0d05-kube-api-access-4m5cr\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:30:51.087002 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:51.086997    4871 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38c81c78-6dca-4ec6-9347-bf512bcc0d05-metrics-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:30:51.087215 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:51.087014    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38c81c78-6dca-4ec6-9347-bf512bcc0d05-maas-api-tls\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:30:51.190393 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:51.190347    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:51.293515 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:51.293411    4871 generic.go:356] "Generic (PLEG): container finished" podID="38c81c78-6dca-4ec6-9347-bf512bcc0d05" containerID="c354fae03e416c897090f452786797e3c3b9ad2824eeadd3b7035297acd1b121" exitCode=0
Oct 30 22:30:51.293515 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:51.293481    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd"
Oct 30 22:30:51.293761 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:51.293495    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd" event={"ID":"38c81c78-6dca-4ec6-9347-bf512bcc0d05","Type":"ContainerDied","Data":"c354fae03e416c897090f452786797e3c3b9ad2824eeadd3b7035297acd1b121"}
Oct 30 22:30:51.293761 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:51.293690    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd" event={"ID":"38c81c78-6dca-4ec6-9347-bf512bcc0d05","Type":"ContainerDied","Data":"389e46bd71d8b8c029c6607570ef3526737eca7d25d0a3dce640d621633d5a43"}
Oct 30 22:30:51.293761 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:51.293730    4871 scope.go:122] "RemoveContainer" containerID="c354fae03e416c897090f452786797e3c3b9ad2824eeadd3b7035297acd1b121"
Oct 30 22:30:51.362499 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:51.356908    4871 scope.go:122] "RemoveContainer" containerID="c354fae03e416c897090f452786797e3c3b9ad2824eeadd3b7035297acd1b121"
Oct 30 22:30:51.362499 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:30:51.357785    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c354fae03e416c897090f452786797e3c3b9ad2824eeadd3b7035297acd1b121\": container with ID starting with c354fae03e416c897090f452786797e3c3b9ad2824eeadd3b7035297acd1b121 not found: ID does not exist" containerID="c354fae03e416c897090f452786797e3c3b9ad2824eeadd3b7035297acd1b121"
Oct 30 22:30:51.362499 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:51.357915    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c354fae03e416c897090f452786797e3c3b9ad2824eeadd3b7035297acd1b121"} err="failed to get container status \"c354fae03e416c897090f452786797e3c3b9ad2824eeadd3b7035297acd1b121\": rpc error: code = NotFound desc = could not find container \"c354fae03e416c897090f452786797e3c3b9ad2824eeadd3b7035297acd1b121\": container with ID starting with c354fae03e416c897090f452786797e3c3b9ad2824eeadd3b7035297acd1b121 not found: ID does not exist"
Oct 30 22:30:51.653933 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:51.615177    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:51.789384 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:51.789343    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd"]
Oct 30 22:30:51.825524 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:51.825490    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:51.848281 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:51.848204    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-7fd427-6d6f5bd775-m5qrd"]
Oct 30 22:30:52.048801 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:52.048703    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="38c81c78-6dca-4ec6-9347-bf512bcc0d05" path="/var/lib/kubelet/pods/38c81c78-6dca-4ec6-9347-bf512bcc0d05/volumes"
Oct 30 22:30:52.158259 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:52.153001    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"]
Oct 30 22:30:52.220868 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:52.220295    4871 cpu_manager.go:414] "containerMap: removing container" podUID="38c81c78-6dca-4ec6-9347-bf512bcc0d05" containerName="maas-api"
Oct 30 22:30:52.223020 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:52.222975    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="38c81c78-6dca-4ec6-9347-bf512bcc0d05" containerName="maas-api"
Oct 30 22:30:52.239061 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:52.223536    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="38c81c78-6dca-4ec6-9347-bf512bcc0d05" containerName="maas-api"
Oct 30 22:30:52.352107 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:52.351570    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:52.397204 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:52.397170    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"]
Oct 30 22:30:52.397525 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:52.397499    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"
Oct 30 22:30:52.454142 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:52.454109    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-ae7a1f-openshift-default-dockercfg-sj7h7\""
Oct 30 22:30:52.459082 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:52.459051    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-credential-socket\") pod \"e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh\" (UID: \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\") " pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"
Oct 30 22:30:52.459316 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:52.459301    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-workload-certs\") pod \"e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh\" (UID: \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\") " pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"
Oct 30 22:30:52.459431 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:52.459421    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-istiod-ca-cert\") pod \"e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh\" (UID: \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\") " pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"
Oct 30 22:30:52.459527 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:52.459518    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-istio-podinfo\") pod \"e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh\" (UID: \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\") " pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"
Oct 30 22:30:52.459724 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:52.459702    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-istio-token\") pod \"e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh\" (UID: \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\") " pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"
Oct 30 22:30:52.459781 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:52.459733    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zv2qk\" (UniqueName: \"kubernetes.io/projected/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-kube-api-access-zv2qk\") pod \"e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh\" (UID: \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\") " pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"
Oct 30 22:30:52.459875 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:52.459863    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-workload-socket\") pod \"e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh\" (UID: \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\") " pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"
Oct 30 22:30:52.459929 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:52.459901    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-istio-envoy\") pod \"e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh\" (UID: \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\") " pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"
Oct 30 22:30:52.459929 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:52.459926    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-istio-data\") pod \"e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh\" (UID: \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\") " pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"
Oct 30 22:30:52.537646 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:52.537164    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:52.573259 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:52.570268    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-workload-socket\") pod \"e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh\" (UID: \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\") " pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"
Oct 30 22:30:52.573259 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:52.570349    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-istio-envoy\") pod \"e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh\" (UID: \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\") " pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"
Oct 30 22:30:52.573259 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:52.570388    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-istio-data\") pod \"e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh\" (UID: \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\") " pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"
Oct 30 22:30:52.573259 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:52.570438    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-credential-socket\") pod \"e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh\" (UID: \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\") " pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"
Oct 30 22:30:52.573259 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:52.570491    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-workload-certs\") pod \"e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh\" (UID: \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\") " pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"
Oct 30 22:30:52.573259 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:52.570525    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-istiod-ca-cert\") pod \"e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh\" (UID: \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\") " pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"
Oct 30 22:30:52.573259 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:52.570593    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-istio-podinfo\") pod \"e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh\" (UID: \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\") " pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"
Oct 30 22:30:52.573259 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:52.570688    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-istio-token\") pod \"e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh\" (UID: \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\") " pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"
Oct 30 22:30:52.573259 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:52.570713    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zv2qk\" (UniqueName: \"kubernetes.io/projected/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-kube-api-access-zv2qk\") pod \"e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh\" (UID: \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\") " pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"
Oct 30 22:30:52.573259 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:52.571602    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-workload-socket\") pod \"e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh\" (UID: \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\") " pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"
Oct 30 22:30:52.591260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:52.585738    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-workload-certs\") pod \"e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh\" (UID: \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\") " pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"
Oct 30 22:30:52.591260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:52.586119    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-istio-data\") pod \"e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh\" (UID: \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\") " pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"
Oct 30 22:30:52.591260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:52.586403    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-credential-socket\") pod \"e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh\" (UID: \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\") " pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"
Oct 30 22:30:52.591591 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:52.591436    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-istiod-ca-cert\") pod \"e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh\" (UID: \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\") " pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"
Oct 30 22:30:52.598793 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:52.598755    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-istio-envoy\") pod \"e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh\" (UID: \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\") " pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"
Oct 30 22:30:52.599682 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:52.599657    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-istio-podinfo\") pod \"e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh\" (UID: \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\") " pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"
Oct 30 22:30:52.638775 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:52.638738    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zv2qk\" (UniqueName: \"kubernetes.io/projected/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-kube-api-access-zv2qk\") pod \"e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh\" (UID: \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\") " pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"
Oct 30 22:30:52.653975 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:52.653941    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-istio-token\") pod \"e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh\" (UID: \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\") " pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"
Oct 30 22:30:52.746939 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:52.746900    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"
Oct 30 22:30:52.924321 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:52.907852    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:53.240100 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:30:53.240046    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod167cb6f4_5f3b_4e79_bd45_3ffc48cefa33.slice/crio-32243cfc36fee7052139aca3db90481be0d62774b92fbc7174a6da4f72fee6d3 WatchSource:0}: Error finding container 32243cfc36fee7052139aca3db90481be0d62774b92fbc7174a6da4f72fee6d3: Status 404 returned error can't find the container with id 32243cfc36fee7052139aca3db90481be0d62774b92fbc7174a6da4f72fee6d3
Oct 30 22:30:53.266477 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:53.265349    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:30:53.266477 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:53.265441    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:30:53.266477 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:53.265490    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:30:53.266477 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:53.266133    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"]
Oct 30 22:30:53.287578 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:53.286448    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:53.455886 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:53.455456    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh" event={"ID":"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33","Type":"ContainerStarted","Data":"32243cfc36fee7052139aca3db90481be0d62774b92fbc7174a6da4f72fee6d3"}
Oct 30 22:30:53.951020 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:53.950987    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:54.465752 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:54.465716    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh" event={"ID":"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33","Type":"ContainerStarted","Data":"66537becb0b66142bd959161daaa772454ddfeab12f96ea2a0b0162044e1b33a"}
Oct 30 22:30:54.652022 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:54.651964    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh" podStartSLOduration=2.65194801 podStartE2EDuration="2.65194801s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:30:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:30:54.64783159 +0000 UTC m=+3196.825320210" watchObservedRunningTime="2026-10-30 22:30:54.65194801 +0000 UTC m=+3196.829436693"
Oct 30 22:30:54.747833 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:54.747739    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"
Oct 30 22:30:54.754215 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:54.754180    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"
Oct 30 22:30:55.475864 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:55.475828    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"
Oct 30 22:30:55.477093 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:55.477068    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"
Oct 30 22:30:56.223412 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:56.223361    4871 patch_prober.go:29] interesting pod/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.176:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:30:56.223575 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:56.223456    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp" podUID="6cbecab8-bfad-40bf-850a-905e4d365d29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.176:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:30:57.591652 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:57.591622    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:57.872376 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:57.872339    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:58.040255 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:58.040205    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:58.197161 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:58.197057    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:58.502295 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:58.502166    4871 generic.go:356] "Generic (PLEG): container finished" podID="6cbecab8-bfad-40bf-850a-905e4d365d29" containerID="a2cfedda456012711321024d33b23567686adf80d7cc9f2cbc8f9f485fbc4d08" exitCode=0
Oct 30 22:30:58.502295 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:58.502220    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp" event={"ID":"6cbecab8-bfad-40bf-850a-905e4d365d29","Type":"ContainerDied","Data":"a2cfedda456012711321024d33b23567686adf80d7cc9f2cbc8f9f485fbc4d08"}
Oct 30 22:30:58.502694 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:58.502675    4871 scope.go:122] "RemoveContainer" containerID="a2cfedda456012711321024d33b23567686adf80d7cc9f2cbc8f9f485fbc4d08"
Oct 30 22:30:58.504321 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:58.504287    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:30:58.504431 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:58.504364    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:30:58.504431 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:58.504397    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:30:59.531679 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:30:59.531641    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp" event={"ID":"6cbecab8-bfad-40bf-850a-905e4d365d29","Type":"ContainerStarted","Data":"915303882d9b027ce1faa0670f764ff80410c5a7e646ad736e9896675b7c9318"}
Oct 30 22:31:00.288736 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:00.288693    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"
Oct 30 22:31:00.288941 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:00.288837    4871 patch_prober.go:29] interesting pod/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.176:15021/healthz/ready\": dial tcp 10.133.0.176:15021: connect: connection refused" start-of-body=
Oct 30 22:31:00.288941 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:00.288886    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp" podUID="6cbecab8-bfad-40bf-850a-905e4d365d29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.176:15021/healthz/ready\": dial tcp 10.133.0.176:15021: connect: connection refused"
Oct 30 22:31:00.461280 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:00.461222    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:00.625944 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:00.625909    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:00.781693 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:00.781660    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"]
Oct 30 22:31:00.781931 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:00.781904    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz" podUID="f4aaa6ee-3ee7-44f4-93ab-673837d4a23b" containerName="istio-proxy" containerID="cri-o://b6d3d7c5aa4164e9e468b3141618f001cd02ffca5fd5df0b287d5ccdac19152b" gracePeriod=30
Oct 30 22:31:00.794537 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:00.794510    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:01.288595 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:01.288556    4871 patch_prober.go:29] interesting pod/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.176:15021/healthz/ready\": dial tcp 10.133.0.176:15021: connect: connection refused" start-of-body=
Oct 30 22:31:01.288780 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:01.288617    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp" podUID="6cbecab8-bfad-40bf-850a-905e4d365d29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.176:15021/healthz/ready\": dial tcp 10.133.0.176:15021: connect: connection refused"
Oct 30 22:31:01.907263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:01.906430    4871 patch_prober.go:29] interesting pod/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:31:01.907263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:01.906501    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz" podUID="f4aaa6ee-3ee7-44f4-93ab-673837d4a23b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:31:02.288545 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:02.288457    4871 patch_prober.go:29] interesting pod/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.176:15021/healthz/ready\": dial tcp 10.133.0.176:15021: connect: connection refused" start-of-body=
Oct 30 22:31:02.288545 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:02.288517    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp" podUID="6cbecab8-bfad-40bf-850a-905e4d365d29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.176:15021/healthz/ready\": dial tcp 10.133.0.176:15021: connect: connection refused"
Oct 30 22:31:02.885100 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:02.885067    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:03.046061 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:03.046015    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:03.352794 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:03.352690    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:03.352794 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:03.352110    4871 patch_prober.go:29] interesting pod/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.176:15021/healthz/ready\": dial tcp 10.133.0.176:15021: connect: connection refused" start-of-body=
Oct 30 22:31:03.353042 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:03.352827    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp" podUID="6cbecab8-bfad-40bf-850a-905e4d365d29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.176:15021/healthz/ready\": dial tcp 10.133.0.176:15021: connect: connection refused"
Oct 30 22:31:04.288620 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:04.288579    4871 patch_prober.go:29] interesting pod/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.176:15021/healthz/ready\": dial tcp 10.133.0.176:15021: connect: connection refused" start-of-body=
Oct 30 22:31:04.289052 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:04.288646    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp" podUID="6cbecab8-bfad-40bf-850a-905e4d365d29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.176:15021/healthz/ready\": dial tcp 10.133.0.176:15021: connect: connection refused"
Oct 30 22:31:05.293137 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:05.293105    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"
Oct 30 22:31:05.293674 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:05.293429    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"
Oct 30 22:31:05.294146 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:05.294130    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"
Oct 30 22:31:05.571010 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:31:05.570927    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:31:06.077502 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:06.077471    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"
Oct 30 22:31:06.112203 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:06.112159    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-istiod-ca-cert\") pod \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\" (UID: \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\") "
Oct 30 22:31:06.112404 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:06.112319    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-workload-socket\") pod \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\" (UID: \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\") "
Oct 30 22:31:06.112523 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:06.112508    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-istio-token\" (UniqueName: \"kubernetes.io/projected/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-istio-token\") pod \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\" (UID: \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\") "
Oct 30 22:31:06.112574 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:06.112558    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-kube-api-access-kl2wq\" (UniqueName: \"kubernetes.io/projected/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-kube-api-access-kl2wq\") pod \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\" (UID: \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\") "
Oct 30 22:31:06.112626 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:06.112602    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-workload-certs\") pod \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\" (UID: \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\") "
Oct 30 22:31:06.112677 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:06.112637    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-istio-data\") pod \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\" (UID: \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\") "
Oct 30 22:31:06.112788 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:06.112680    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-credential-socket\") pod \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\" (UID: \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\") "
Oct 30 22:31:06.112788 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:06.112764    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-istio-podinfo\") pod \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\" (UID: \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\") "
Oct 30 22:31:06.112908 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:06.112800    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-istio-envoy\") pod \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\" (UID: \"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b\") "
Oct 30 22:31:06.113058 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:06.113029    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-credential-socket" pod "f4aaa6ee-3ee7-44f4-93ab-673837d4a23b" (UID: "f4aaa6ee-3ee7-44f4-93ab-673837d4a23b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:06.113468 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:06.113180    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-workload-certs" pod "f4aaa6ee-3ee7-44f4-93ab-673837d4a23b" (UID: "f4aaa6ee-3ee7-44f4-93ab-673837d4a23b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:06.113468 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:06.113208    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-istio-data" pod "f4aaa6ee-3ee7-44f4-93ab-673837d4a23b" (UID: "f4aaa6ee-3ee7-44f4-93ab-673837d4a23b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:06.113468 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:06.113419    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:31:06.113468 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:06.113441    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:31:06.113468 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:06.113463    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:31:06.113827 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:06.112649    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-workload-socket" pod "f4aaa6ee-3ee7-44f4-93ab-673837d4a23b" (UID: "f4aaa6ee-3ee7-44f4-93ab-673837d4a23b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:06.113827 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:06.112576    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-istiod-ca-cert" pod "f4aaa6ee-3ee7-44f4-93ab-673837d4a23b" (UID: "f4aaa6ee-3ee7-44f4-93ab-673837d4a23b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:31:06.115287 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:06.115220    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-istio-token" pod "f4aaa6ee-3ee7-44f4-93ab-673837d4a23b" (UID: "f4aaa6ee-3ee7-44f4-93ab-673837d4a23b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:31:06.115466 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:06.115438    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-istio-podinfo" pod "f4aaa6ee-3ee7-44f4-93ab-673837d4a23b" (UID: "f4aaa6ee-3ee7-44f4-93ab-673837d4a23b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:31:06.116025 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:06.115998    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-istio-envoy" pod "f4aaa6ee-3ee7-44f4-93ab-673837d4a23b" (UID: "f4aaa6ee-3ee7-44f4-93ab-673837d4a23b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:06.116123 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:06.116085    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-kube-api-access-kl2wq" pod "f4aaa6ee-3ee7-44f4-93ab-673837d4a23b" (UID: "f4aaa6ee-3ee7-44f4-93ab-673837d4a23b"). InnerVolumeSpecName "kube-api-access-kl2wq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:31:06.215229 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:06.215150    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:31:06.215229 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:06.215184    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:31:06.215229 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:06.215193    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:31:06.215229 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:06.215202    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:31:06.215229 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:06.215212    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:31:06.215229 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:06.215221    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kl2wq\" (UniqueName: \"kubernetes.io/projected/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b-kube-api-access-kl2wq\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:31:06.805645 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:06.805597    4871 generic.go:356] "Generic (PLEG): container finished" podID="f4aaa6ee-3ee7-44f4-93ab-673837d4a23b" containerID="b6d3d7c5aa4164e9e468b3141618f001cd02ffca5fd5df0b287d5ccdac19152b" exitCode=0
Oct 30 22:31:06.806161 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:06.805653    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"
Oct 30 22:31:06.806161 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:06.805675    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz" event={"ID":"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b","Type":"ContainerDied","Data":"b6d3d7c5aa4164e9e468b3141618f001cd02ffca5fd5df0b287d5ccdac19152b"}
Oct 30 22:31:06.806161 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:06.805739    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz" event={"ID":"f4aaa6ee-3ee7-44f4-93ab-673837d4a23b","Type":"ContainerDied","Data":"d6dd37bebac94eaf967c4351cee1bb278601b10b4cfa9bc70a4a75aba9b33f45"}
Oct 30 22:31:06.806161 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:06.805771    4871 scope.go:122] "RemoveContainer" containerID="b6d3d7c5aa4164e9e468b3141618f001cd02ffca5fd5df0b287d5ccdac19152b"
Oct 30 22:31:06.823978 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:06.823957    4871 scope.go:122] "RemoveContainer" containerID="446a7c884a4280e9cfbcd0dfc303c3f87bafa507bc5c657fde1ecfeba6339ab8"
Oct 30 22:31:06.836582 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:06.836494    4871 scope.go:122] "RemoveContainer" containerID="b6d3d7c5aa4164e9e468b3141618f001cd02ffca5fd5df0b287d5ccdac19152b"
Oct 30 22:31:06.836900 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:31:06.836876    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6d3d7c5aa4164e9e468b3141618f001cd02ffca5fd5df0b287d5ccdac19152b\": container with ID starting with b6d3d7c5aa4164e9e468b3141618f001cd02ffca5fd5df0b287d5ccdac19152b not found: ID does not exist" containerID="b6d3d7c5aa4164e9e468b3141618f001cd02ffca5fd5df0b287d5ccdac19152b"
Oct 30 22:31:06.837001 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:06.836908    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6d3d7c5aa4164e9e468b3141618f001cd02ffca5fd5df0b287d5ccdac19152b"} err="failed to get container status \"b6d3d7c5aa4164e9e468b3141618f001cd02ffca5fd5df0b287d5ccdac19152b\": rpc error: code = NotFound desc = could not find container \"b6d3d7c5aa4164e9e468b3141618f001cd02ffca5fd5df0b287d5ccdac19152b\": container with ID starting with b6d3d7c5aa4164e9e468b3141618f001cd02ffca5fd5df0b287d5ccdac19152b not found: ID does not exist"
Oct 30 22:31:06.837001 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:06.836927    4871 scope.go:122] "RemoveContainer" containerID="446a7c884a4280e9cfbcd0dfc303c3f87bafa507bc5c657fde1ecfeba6339ab8"
Oct 30 22:31:06.837261 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:31:06.837227    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"446a7c884a4280e9cfbcd0dfc303c3f87bafa507bc5c657fde1ecfeba6339ab8\": container with ID starting with 446a7c884a4280e9cfbcd0dfc303c3f87bafa507bc5c657fde1ecfeba6339ab8 not found: ID does not exist" containerID="446a7c884a4280e9cfbcd0dfc303c3f87bafa507bc5c657fde1ecfeba6339ab8"
Oct 30 22:31:06.837323 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:06.837271    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"446a7c884a4280e9cfbcd0dfc303c3f87bafa507bc5c657fde1ecfeba6339ab8"} err="failed to get container status \"446a7c884a4280e9cfbcd0dfc303c3f87bafa507bc5c657fde1ecfeba6339ab8\": rpc error: code = NotFound desc = could not find container \"446a7c884a4280e9cfbcd0dfc303c3f87bafa507bc5c657fde1ecfeba6339ab8\": container with ID starting with 446a7c884a4280e9cfbcd0dfc303c3f87bafa507bc5c657fde1ecfeba6339ab8 not found: ID does not exist"
Oct 30 22:31:06.988153 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:06.988113    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"]
Oct 30 22:31:07.042534 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:07.042488    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-b-7fd427-openshift-default-86d6585d9c-v2cbz"]
Oct 30 22:31:07.963420 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:07.963386    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f4aaa6ee-3ee7-44f4-93ab-673837d4a23b" path="/var/lib/kubelet/pods/f4aaa6ee-3ee7-44f4-93ab-673837d4a23b/volumes"
Oct 30 22:31:07.997968 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:07.997922    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:10.708603 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:10.708569    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:11.194852 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:11.194815    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:12.035201 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.035149    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"]
Oct 30 22:31:12.036416 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.036111    4871 cpu_manager.go:414] "containerMap: removing container" podUID="f4aaa6ee-3ee7-44f4-93ab-673837d4a23b" containerName="istio-proxy"
Oct 30 22:31:12.036416 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.036140    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f4aaa6ee-3ee7-44f4-93ab-673837d4a23b" containerName="istio-proxy"
Oct 30 22:31:12.036416 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.036356    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="f4aaa6ee-3ee7-44f4-93ab-673837d4a23b" containerName="istio-proxy"
Oct 30 22:31:12.036416 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.036376    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="f4aaa6ee-3ee7-44f4-93ab-673837d4a23b" containerName="istio-proxy"
Oct 30 22:31:12.047307 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.046682    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"
Oct 30 22:31:12.116101 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.110660    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-workload-certs\") pod \"e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5\" (UID: \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\") " pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"
Oct 30 22:31:12.116101 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.110793    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-istio-data\") pod \"e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5\" (UID: \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\") " pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"
Oct 30 22:31:12.116101 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.110826    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-istio-token\") pod \"e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5\" (UID: \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\") " pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"
Oct 30 22:31:12.116101 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.110950    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-istio-podinfo\") pod \"e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5\" (UID: \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\") " pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"
Oct 30 22:31:12.116101 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.111040    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-credential-socket\") pod \"e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5\" (UID: \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\") " pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"
Oct 30 22:31:12.116101 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.111175    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-istiod-ca-cert\") pod \"e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5\" (UID: \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\") " pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"
Oct 30 22:31:12.116101 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.111304    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkh5j\" (UniqueName: \"kubernetes.io/projected/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-kube-api-access-hkh5j\") pod \"e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5\" (UID: \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\") " pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"
Oct 30 22:31:12.116101 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.112803    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-istio-envoy\") pod \"e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5\" (UID: \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\") " pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"
Oct 30 22:31:12.116101 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.112873    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-workload-socket\") pod \"e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5\" (UID: \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\") " pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"
Oct 30 22:31:12.140395 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.140354    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w0-c29159-openshift-default-dockercfg-6rx4f\""
Oct 30 22:31:12.146516 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.146458    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:12.166734 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.166659    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"]
Oct 30 22:31:12.225570 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.225520    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-workload-certs\") pod \"e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5\" (UID: \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\") " pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"
Oct 30 22:31:12.226473 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.226323    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-workload-certs\") pod \"e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5\" (UID: \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\") " pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"
Oct 30 22:31:12.226706 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.226685    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-istio-data\") pod \"e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5\" (UID: \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\") " pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"
Oct 30 22:31:12.227154 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.227136    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-istio-token\") pod \"e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5\" (UID: \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\") " pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"
Oct 30 22:31:12.227669 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.227086    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-istio-data\") pod \"e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5\" (UID: \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\") " pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"
Oct 30 22:31:12.227855 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.227836    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-istio-podinfo\") pod \"e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5\" (UID: \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\") " pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"
Oct 30 22:31:12.228544 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.228524    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-credential-socket\") pod \"e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5\" (UID: \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\") " pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"
Oct 30 22:31:12.229061 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.228953    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-credential-socket\") pod \"e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5\" (UID: \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\") " pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"
Oct 30 22:31:12.229940 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.229919    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-istiod-ca-cert\") pod \"e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5\" (UID: \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\") " pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"
Oct 30 22:31:12.230106 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.230088    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-istiod-ca-cert\") pod \"e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5\" (UID: \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\") " pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"
Oct 30 22:31:12.230566 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.230225    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkh5j\" (UniqueName: \"kubernetes.io/projected/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-kube-api-access-hkh5j\") pod \"e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5\" (UID: \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\") " pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"
Oct 30 22:31:12.230950 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.230923    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-istio-envoy\") pod \"e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5\" (UID: \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\") " pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"
Oct 30 22:31:12.231782 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.231755    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-workload-socket\") pod \"e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5\" (UID: \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\") " pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"
Oct 30 22:31:12.232535 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.232361    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-workload-socket\") pod \"e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5\" (UID: \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\") " pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"
Oct 30 22:31:12.237152 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.237104    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-istio-podinfo\") pod \"e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5\" (UID: \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\") " pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"
Oct 30 22:31:12.278688 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.278647    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-istio-envoy\") pod \"e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5\" (UID: \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\") " pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"
Oct 30 22:31:12.294896 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.293793    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-istio-token\") pod \"e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5\" (UID: \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\") " pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"
Oct 30 22:31:12.295771 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.295738    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkh5j\" (UniqueName: \"kubernetes.io/projected/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-kube-api-access-hkh5j\") pod \"e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5\" (UID: \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\") " pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"
Oct 30 22:31:12.361068 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.359834    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:12.392419 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.392384    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"
Oct 30 22:31:12.903998 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:31:12.903956    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8acc9cc7_ebae_4d92_8e8c_c8ed8e615030.slice/crio-55d6a8ebf9a2c233ce1633ea64d5fda9fd51c3bae3d354034ea5e816e37c881d WatchSource:0}: Error finding container 55d6a8ebf9a2c233ce1633ea64d5fda9fd51c3bae3d354034ea5e816e37c881d: Status 404 returned error can't find the container with id 55d6a8ebf9a2c233ce1633ea64d5fda9fd51c3bae3d354034ea5e816e37c881d
Oct 30 22:31:12.907771 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.907735    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:31:12.907895 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.907821    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:31:12.907895 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.907868    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:31:12.910188 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.910161    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"]
Oct 30 22:31:12.917914 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:12.917886    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:13.882273 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:13.882217    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5" event={"ID":"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030","Type":"ContainerStarted","Data":"3214dd3b7a2df7d71e4ac79864cad8403ab0b9eb2556519fbd6f2d6fe52d17cd"}
Oct 30 22:31:13.882780 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:13.882301    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5" event={"ID":"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030","Type":"ContainerStarted","Data":"55d6a8ebf9a2c233ce1633ea64d5fda9fd51c3bae3d354034ea5e816e37c881d"}
Oct 30 22:31:14.063223 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:14.063168    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5" podStartSLOduration=3.063154306 podStartE2EDuration="3.063154306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:31:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:31:14.060105098 +0000 UTC m=+3216.237593715" watchObservedRunningTime="2026-10-30 22:31:14.063154306 +0000 UTC m=+3216.240642923"
Oct 30 22:31:14.393227 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:14.393187    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"
Oct 30 22:31:14.399347 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:14.399316    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"
Oct 30 22:31:14.890498 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:14.890461    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"
Oct 30 22:31:14.891349 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:14.891331    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"
Oct 30 22:31:16.753389 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:16.753352    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:16.935340 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:16.935298    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:17.102555 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:17.102462    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:17.912191 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:17.912156    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:18.230728 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:18.230616    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:18.701589 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:18.701557    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:18.958129 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:18.958046    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:19.277793 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:19.277692    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"]
Oct 30 22:31:19.278376 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:19.278360    4871 cpu_manager.go:414] "containerMap: removing container" podUID="f4aaa6ee-3ee7-44f4-93ab-673837d4a23b" containerName="istio-proxy"
Oct 30 22:31:19.278438 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:19.278376    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f4aaa6ee-3ee7-44f4-93ab-673837d4a23b" containerName="istio-proxy"
Oct 30 22:31:19.283452 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:19.283428    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"
Oct 30 22:31:19.341451 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:19.341416    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxt85\" (UniqueName: \"kubernetes.io/projected/b873ef79-e292-4496-a9ec-8d2ec09f65f6-kube-api-access-xxt85\") pod \"e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r\" (UID: \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\") " pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"
Oct 30 22:31:19.341652 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:19.341474    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b873ef79-e292-4496-a9ec-8d2ec09f65f6-istio-envoy\") pod \"e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r\" (UID: \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\") " pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"
Oct 30 22:31:19.341652 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:19.341507    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b873ef79-e292-4496-a9ec-8d2ec09f65f6-istiod-ca-cert\") pod \"e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r\" (UID: \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\") " pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"
Oct 30 22:31:19.341652 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:19.341549    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"]
Oct 30 22:31:19.341780 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:19.341703    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b873ef79-e292-4496-a9ec-8d2ec09f65f6-istio-podinfo\") pod \"e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r\" (UID: \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\") " pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"
Oct 30 22:31:19.341865 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:19.341845    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b873ef79-e292-4496-a9ec-8d2ec09f65f6-istio-token\") pod \"e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r\" (UID: \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\") " pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"
Oct 30 22:31:19.341906 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:19.341890    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b873ef79-e292-4496-a9ec-8d2ec09f65f6-istio-data\") pod \"e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r\" (UID: \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\") " pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"
Oct 30 22:31:19.342026 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:19.342006    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b873ef79-e292-4496-a9ec-8d2ec09f65f6-workload-certs\") pod \"e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r\" (UID: \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\") " pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"
Oct 30 22:31:19.342074 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:19.342040    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b873ef79-e292-4496-a9ec-8d2ec09f65f6-workload-socket\") pod \"e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r\" (UID: \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\") " pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"
Oct 30 22:31:19.342169 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:19.342148    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b873ef79-e292-4496-a9ec-8d2ec09f65f6-credential-socket\") pod \"e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r\" (UID: \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\") " pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"
Oct 30 22:31:19.370690 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:19.370653    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:19.387534 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:19.387503    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w1-323082-openshift-default-dockercfg-jrzlk\""
Oct 30 22:31:19.447267 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:19.445912    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b873ef79-e292-4496-a9ec-8d2ec09f65f6-credential-socket\") pod \"e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r\" (UID: \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\") " pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"
Oct 30 22:31:19.447267 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:19.446052    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxt85\" (UniqueName: \"kubernetes.io/projected/b873ef79-e292-4496-a9ec-8d2ec09f65f6-kube-api-access-xxt85\") pod \"e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r\" (UID: \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\") " pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"
Oct 30 22:31:19.447267 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:19.446102    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b873ef79-e292-4496-a9ec-8d2ec09f65f6-istio-envoy\") pod \"e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r\" (UID: \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\") " pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"
Oct 30 22:31:19.447267 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:19.446151    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b873ef79-e292-4496-a9ec-8d2ec09f65f6-istiod-ca-cert\") pod \"e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r\" (UID: \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\") " pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"
Oct 30 22:31:19.447267 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:19.446226    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b873ef79-e292-4496-a9ec-8d2ec09f65f6-istio-podinfo\") pod \"e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r\" (UID: \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\") " pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"
Oct 30 22:31:19.456272 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:19.455118    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b873ef79-e292-4496-a9ec-8d2ec09f65f6-credential-socket\") pod \"e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r\" (UID: \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\") " pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"
Oct 30 22:31:19.536321 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:19.523538    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b873ef79-e292-4496-a9ec-8d2ec09f65f6-istio-token\") pod \"e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r\" (UID: \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\") " pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"
Oct 30 22:31:19.536321 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:19.523648    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b873ef79-e292-4496-a9ec-8d2ec09f65f6-istio-data\") pod \"e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r\" (UID: \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\") " pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"
Oct 30 22:31:19.536321 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:19.523787    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b873ef79-e292-4496-a9ec-8d2ec09f65f6-workload-certs\") pod \"e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r\" (UID: \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\") " pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"
Oct 30 22:31:19.536321 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:19.523823    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b873ef79-e292-4496-a9ec-8d2ec09f65f6-workload-socket\") pod \"e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r\" (UID: \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\") " pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"
Oct 30 22:31:19.536321 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:19.524344    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b873ef79-e292-4496-a9ec-8d2ec09f65f6-workload-socket\") pod \"e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r\" (UID: \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\") " pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"
Oct 30 22:31:19.536321 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:19.526142    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b873ef79-e292-4496-a9ec-8d2ec09f65f6-istiod-ca-cert\") pod \"e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r\" (UID: \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\") " pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"
Oct 30 22:31:19.565324 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:19.556743    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b873ef79-e292-4496-a9ec-8d2ec09f65f6-workload-certs\") pod \"e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r\" (UID: \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\") " pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"
Oct 30 22:31:19.565324 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:19.557083    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b873ef79-e292-4496-a9ec-8d2ec09f65f6-istio-data\") pod \"e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r\" (UID: \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\") " pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"
Oct 30 22:31:19.589269 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:19.588302    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b873ef79-e292-4496-a9ec-8d2ec09f65f6-istio-podinfo\") pod \"e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r\" (UID: \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\") " pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"
Oct 30 22:31:19.608260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:19.605181    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:19.667675 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:19.667601    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b873ef79-e292-4496-a9ec-8d2ec09f65f6-istio-envoy\") pod \"e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r\" (UID: \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\") " pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"
Oct 30 22:31:19.667932 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:19.667709    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxt85\" (UniqueName: \"kubernetes.io/projected/b873ef79-e292-4496-a9ec-8d2ec09f65f6-kube-api-access-xxt85\") pod \"e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r\" (UID: \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\") " pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"
Oct 30 22:31:19.684882 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:19.684806    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b873ef79-e292-4496-a9ec-8d2ec09f65f6-istio-token\") pod \"e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r\" (UID: \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\") " pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"
Oct 30 22:31:20.029973 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:20.029933    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"
Oct 30 22:31:20.032220 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:20.032186    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:20.256277 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:20.252000    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:20.762110 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:31:20.762076    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb873ef79_e292_4496_a9ec_8d2ec09f65f6.slice/crio-787047c924b6ea60881427ecc59565dd1c9380ab7585f2918ce7977afb310cad WatchSource:0}: Error finding container 787047c924b6ea60881427ecc59565dd1c9380ab7585f2918ce7977afb310cad: Status 404 returned error can't find the container with id 787047c924b6ea60881427ecc59565dd1c9380ab7585f2918ce7977afb310cad
Oct 30 22:31:20.769311 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:20.764278    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"]
Oct 30 22:31:20.769541 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:20.767189    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:31:20.770138 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:20.770105    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:31:20.770375 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:20.770352    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:31:21.041381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:21.041272    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r" event={"ID":"b873ef79-e292-4496-a9ec-8d2ec09f65f6","Type":"ContainerStarted","Data":"23e5724be2c58f004dd3c8be76117e3f3fa9d18f096a485d1a48c673db63e0f6"}
Oct 30 22:31:21.041381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:21.041337    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r" event={"ID":"b873ef79-e292-4496-a9ec-8d2ec09f65f6","Type":"ContainerStarted","Data":"787047c924b6ea60881427ecc59565dd1c9380ab7585f2918ce7977afb310cad"}
Oct 30 22:31:21.177550 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:21.177487    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r" podStartSLOduration=2.177471006 podStartE2EDuration="2.177471006s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:31:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:31:21.176437083 +0000 UTC m=+3223.353925702" watchObservedRunningTime="2026-10-30 22:31:21.177471006 +0000 UTC m=+3223.354959676"
Oct 30 22:31:22.031140 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:22.031108    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"
Oct 30 22:31:22.037173 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:22.037146    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"
Oct 30 22:31:22.049151 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:22.049122    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"
Oct 30 22:31:22.049993 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:22.049976    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"
Oct 30 22:31:23.369899 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:23.369848    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:23.576592 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:23.576554    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:23.795300 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:23.795162    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:24.032166 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:24.032125    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:26.842068 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:26.842016    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:27.081937 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:27.081892    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:28.656828 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:28.656790    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:34.465838 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:34.465791    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:34.643174 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:34.643129    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:35.569586 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:31:35.569542    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:31:37.938504 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:37.938466    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:38.154903 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:38.154839    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:38.266520 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:38.266429    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh"]
Oct 30 22:31:38.272231 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:38.272198    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh"
Oct 30 22:31:38.300277 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:38.300202    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dbb05364-f153-4407-be11-808adf3eb888-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh\" (UID: \"dbb05364-f153-4407-be11-808adf3eb888\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh"
Oct 30 22:31:38.300491 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:38.300453    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rz52\" (UniqueName: \"kubernetes.io/projected/dbb05364-f153-4407-be11-808adf3eb888-kube-api-access-9rz52\") pod \"maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh\" (UID: \"dbb05364-f153-4407-be11-808adf3eb888\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh"
Oct 30 22:31:38.300585 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:38.300565    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dbb05364-f153-4407-be11-808adf3eb888-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh\" (UID: \"dbb05364-f153-4407-be11-808adf3eb888\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh"
Oct 30 22:31:38.300638 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:38.300626    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbb05364-f153-4407-be11-808adf3eb888-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh\" (UID: \"dbb05364-f153-4407-be11-808adf3eb888\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh"
Oct 30 22:31:38.329090 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:38.329048    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh"]
Oct 30 22:31:38.376248 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:38.376209    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-5bb0b1-serving-cert\""
Oct 30 22:31:38.401755 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:38.401723    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rz52\" (UniqueName: \"kubernetes.io/projected/dbb05364-f153-4407-be11-808adf3eb888-kube-api-access-9rz52\") pod \"maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh\" (UID: \"dbb05364-f153-4407-be11-808adf3eb888\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh"
Oct 30 22:31:38.401955 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:38.401794    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dbb05364-f153-4407-be11-808adf3eb888-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh\" (UID: \"dbb05364-f153-4407-be11-808adf3eb888\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh"
Oct 30 22:31:38.401955 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:38.401844    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbb05364-f153-4407-be11-808adf3eb888-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh\" (UID: \"dbb05364-f153-4407-be11-808adf3eb888\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh"
Oct 30 22:31:38.402084 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:38.401967    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dbb05364-f153-4407-be11-808adf3eb888-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh\" (UID: \"dbb05364-f153-4407-be11-808adf3eb888\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh"
Oct 30 22:31:38.402200 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:31:38.402171    4871 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5bb0b1-serving-cert: secret "maas-api-e2e-shared-b-w2-5bb0b1-serving-cert" not found
Oct 30 22:31:38.402318 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:31:38.402296    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbb05364-f153-4407-be11-808adf3eb888-maas-api-tls podName:dbb05364-f153-4407-be11-808adf3eb888 nodeName:}" failed. No retries permitted until 2026-10-30 22:31:38.902269996 +0000 UTC m=+3241.079758614 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/dbb05364-f153-4407-be11-808adf3eb888-maas-api-tls") pod "maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh" (UID: "dbb05364-f153-4407-be11-808adf3eb888") : secret "maas-api-e2e-shared-b-w2-5bb0b1-serving-cert" not found
Oct 30 22:31:38.403337 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:38.403309    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbb05364-f153-4407-be11-808adf3eb888-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh\" (UID: \"dbb05364-f153-4407-be11-808adf3eb888\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh"
Oct 30 22:31:38.404915 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:38.404890    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dbb05364-f153-4407-be11-808adf3eb888-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh\" (UID: \"dbb05364-f153-4407-be11-808adf3eb888\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh"
Oct 30 22:31:38.462522 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:38.462485    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:38.465827 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:38.465796    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rz52\" (UniqueName: \"kubernetes.io/projected/dbb05364-f153-4407-be11-808adf3eb888-kube-api-access-9rz52\") pod \"maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh\" (UID: \"dbb05364-f153-4407-be11-808adf3eb888\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh"
Oct 30 22:31:38.652881 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:38.652837    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:38.921348 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:38.921221    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:38.965405 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:38.965361    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dbb05364-f153-4407-be11-808adf3eb888-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh\" (UID: \"dbb05364-f153-4407-be11-808adf3eb888\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh"
Oct 30 22:31:38.968189 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:38.968157    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dbb05364-f153-4407-be11-808adf3eb888-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh\" (UID: \"dbb05364-f153-4407-be11-808adf3eb888\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh"
Oct 30 22:31:39.186843 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:39.186747    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh"
Oct 30 22:31:39.591603 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:39.591563    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh"]
Oct 30 22:31:40.234582 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:40.234541    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh" event={"ID":"dbb05364-f153-4407-be11-808adf3eb888","Type":"ContainerStarted","Data":"80317366d900ec47f3eae7f3b16bc05668f1483c777790ba0de99588b7010a69"}
Oct 30 22:31:41.299270 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:41.299148    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh" event={"ID":"dbb05364-f153-4407-be11-808adf3eb888","Type":"ContainerStarted","Data":"2f9b6d5d2e045cbd96f52f029f9bf66479487a406fd2a6479f00e5f79361f329"}
Oct 30 22:31:41.301206 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:41.301121    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh"
Oct 30 22:31:41.426106 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:41.426048    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh" podStartSLOduration=2.418118288 podStartE2EDuration="3.42603472s" totalImagesPullingTime="1.007916432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:31:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:31:41.423844508 +0000 UTC m=+3243.601333125" watchObservedRunningTime="2026-10-30 22:31:41.42603472 +0000 UTC m=+3243.603523328"
Oct 30 22:31:43.298928 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:43.298893    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:47.188739 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:47.188702    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:47.573178 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:47.573086    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:47.855472 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:47.855390    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:48.352682 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:48.352645    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh"
Oct 30 22:31:48.670829 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:48.670790    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:52.350667 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:52.350624    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:52.509554 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:52.509488    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:52.659354 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:52.659321    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:57.188183 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:31:57.188141    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:01.805672 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:01.805638    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:05.571319 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:32:05.571277    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:32:06.055146 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:06.055112    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:06.522282 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:06.522222    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:11.026086 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:11.026051    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:15.498926 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:15.498893    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:16.292574 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:16.292517    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:20.336461 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:20.336364    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:24.885742 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:24.885697    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-355213-6bbf99fb98-l6847"]
Oct 30 22:32:24.890782 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:24.890758    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-355213-6bbf99fb98-l6847"
Oct 30 22:32:24.947007 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:24.946966    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-355213-6bbf99fb98-l6847"]
Oct 30 22:32:24.976891 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:24.976843    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b78eced2-5755-49b5-9666-3af5f738c437-metrics-certs\") pod \"maas-api-e2e-explicit-355213-6bbf99fb98-l6847\" (UID: \"b78eced2-5755-49b5-9666-3af5f738c437\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-355213-6bbf99fb98-l6847"
Oct 30 22:32:24.977105 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:24.977082    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b78eced2-5755-49b5-9666-3af5f738c437-openshift-service-ca\") pod \"maas-api-e2e-explicit-355213-6bbf99fb98-l6847\" (UID: \"b78eced2-5755-49b5-9666-3af5f738c437\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-355213-6bbf99fb98-l6847"
Oct 30 22:32:24.977219 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:24.977194    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbfhp\" (UniqueName: \"kubernetes.io/projected/b78eced2-5755-49b5-9666-3af5f738c437-kube-api-access-cbfhp\") pod \"maas-api-e2e-explicit-355213-6bbf99fb98-l6847\" (UID: \"b78eced2-5755-49b5-9666-3af5f738c437\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-355213-6bbf99fb98-l6847"
Oct 30 22:32:24.977323 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:24.977264    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b78eced2-5755-49b5-9666-3af5f738c437-maas-api-tls\") pod \"maas-api-e2e-explicit-355213-6bbf99fb98-l6847\" (UID: \"b78eced2-5755-49b5-9666-3af5f738c437\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-355213-6bbf99fb98-l6847"
Oct 30 22:32:24.993033 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:24.993002    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-355213-serving-cert\""
Oct 30 22:32:25.079167 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:25.079116    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b78eced2-5755-49b5-9666-3af5f738c437-metrics-certs\") pod \"maas-api-e2e-explicit-355213-6bbf99fb98-l6847\" (UID: \"b78eced2-5755-49b5-9666-3af5f738c437\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-355213-6bbf99fb98-l6847"
Oct 30 22:32:25.079414 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:25.079268    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b78eced2-5755-49b5-9666-3af5f738c437-openshift-service-ca\") pod \"maas-api-e2e-explicit-355213-6bbf99fb98-l6847\" (UID: \"b78eced2-5755-49b5-9666-3af5f738c437\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-355213-6bbf99fb98-l6847"
Oct 30 22:32:25.079414 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:25.079309    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbfhp\" (UniqueName: \"kubernetes.io/projected/b78eced2-5755-49b5-9666-3af5f738c437-kube-api-access-cbfhp\") pod \"maas-api-e2e-explicit-355213-6bbf99fb98-l6847\" (UID: \"b78eced2-5755-49b5-9666-3af5f738c437\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-355213-6bbf99fb98-l6847"
Oct 30 22:32:25.079414 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:25.079337    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b78eced2-5755-49b5-9666-3af5f738c437-maas-api-tls\") pod \"maas-api-e2e-explicit-355213-6bbf99fb98-l6847\" (UID: \"b78eced2-5755-49b5-9666-3af5f738c437\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-355213-6bbf99fb98-l6847"
Oct 30 22:32:25.079611 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:32:25.079439    4871 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-explicit-355213-serving-cert: secret "maas-api-e2e-explicit-355213-serving-cert" not found
Oct 30 22:32:25.079611 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:32:25.079493    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b78eced2-5755-49b5-9666-3af5f738c437-maas-api-tls podName:b78eced2-5755-49b5-9666-3af5f738c437 nodeName:}" failed. No retries permitted until 2026-10-30 22:32:25.579477005 +0000 UTC m=+3287.756965599 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b78eced2-5755-49b5-9666-3af5f738c437-maas-api-tls") pod "maas-api-e2e-explicit-355213-6bbf99fb98-l6847" (UID: "b78eced2-5755-49b5-9666-3af5f738c437") : secret "maas-api-e2e-explicit-355213-serving-cert" not found
Oct 30 22:32:25.080035 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:25.080007    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b78eced2-5755-49b5-9666-3af5f738c437-openshift-service-ca\") pod \"maas-api-e2e-explicit-355213-6bbf99fb98-l6847\" (UID: \"b78eced2-5755-49b5-9666-3af5f738c437\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-355213-6bbf99fb98-l6847"
Oct 30 22:32:25.081694 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:25.081672    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b78eced2-5755-49b5-9666-3af5f738c437-metrics-certs\") pod \"maas-api-e2e-explicit-355213-6bbf99fb98-l6847\" (UID: \"b78eced2-5755-49b5-9666-3af5f738c437\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-355213-6bbf99fb98-l6847"
Oct 30 22:32:25.092075 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:25.092048    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:25.173331 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:25.173165    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbfhp\" (UniqueName: \"kubernetes.io/projected/b78eced2-5755-49b5-9666-3af5f738c437-kube-api-access-cbfhp\") pod \"maas-api-e2e-explicit-355213-6bbf99fb98-l6847\" (UID: \"b78eced2-5755-49b5-9666-3af5f738c437\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-355213-6bbf99fb98-l6847"
Oct 30 22:32:25.293679 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:25.293640    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:25.466833 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:25.466713    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:25.593675 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:25.593635    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b78eced2-5755-49b5-9666-3af5f738c437-maas-api-tls\") pod \"maas-api-e2e-explicit-355213-6bbf99fb98-l6847\" (UID: \"b78eced2-5755-49b5-9666-3af5f738c437\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-355213-6bbf99fb98-l6847"
Oct 30 22:32:25.596251 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:25.596218    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b78eced2-5755-49b5-9666-3af5f738c437-maas-api-tls\") pod \"maas-api-e2e-explicit-355213-6bbf99fb98-l6847\" (UID: \"b78eced2-5755-49b5-9666-3af5f738c437\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-355213-6bbf99fb98-l6847"
Oct 30 22:32:25.811588 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:25.811495    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-355213-6bbf99fb98-l6847"
Oct 30 22:32:26.227931 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:26.227892    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-355213-6bbf99fb98-l6847"]
Oct 30 22:32:26.230687 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:32:26.230652    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb78eced2_5755_49b5_9666_3af5f738c437.slice/crio-045994d17792941366e0b74b899ae88a94015ff2713ecfd67e0aa0c582fbe94c WatchSource:0}: Error finding container 045994d17792941366e0b74b899ae88a94015ff2713ecfd67e0aa0c582fbe94c: Status 404 returned error can't find the container with id 045994d17792941366e0b74b899ae88a94015ff2713ecfd67e0aa0c582fbe94c
Oct 30 22:32:26.775188 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:26.775147    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-355213-6bbf99fb98-l6847" event={"ID":"b78eced2-5755-49b5-9666-3af5f738c437","Type":"ContainerStarted","Data":"045994d17792941366e0b74b899ae88a94015ff2713ecfd67e0aa0c582fbe94c"}
Oct 30 22:32:27.786784 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:27.786744    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-355213-6bbf99fb98-l6847" event={"ID":"b78eced2-5755-49b5-9666-3af5f738c437","Type":"ContainerStarted","Data":"dbc4ab4e324853931dbaefd112d936b6e948797777866dc81399a01634b9def2"}
Oct 30 22:32:27.787334 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:27.786948    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-355213-6bbf99fb98-l6847"
Oct 30 22:32:30.109958 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:30.109925    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:30.818512 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:30.818478    4871 generic.go:356] "Generic (PLEG): container finished" podID="167cb6f4-5f3b-4e79-bd45-3ffc48cefa33" containerID="66537becb0b66142bd959161daaa772454ddfeab12f96ea2a0b0162044e1b33a" exitCode=0
Oct 30 22:32:30.818748 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:30.818551    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh" event={"ID":"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33","Type":"ContainerDied","Data":"66537becb0b66142bd959161daaa772454ddfeab12f96ea2a0b0162044e1b33a"}
Oct 30 22:32:30.819099 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:30.819079    4871 scope.go:122] "RemoveContainer" containerID="66537becb0b66142bd959161daaa772454ddfeab12f96ea2a0b0162044e1b33a"
Oct 30 22:32:30.820919 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:30.820891    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:32:30.821007 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:30.820967    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:32:30.821007 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:30.820999    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:32:31.036881 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:31.036808    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-355213-6bbf99fb98-l6847" podStartSLOduration=6.114799527 podStartE2EDuration="7.036787881s" totalImagesPullingTime="921.988354ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:32:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:32:27.929928571 +0000 UTC m=+3290.107417191" watchObservedRunningTime="2026-10-30 22:32:31.036787881 +0000 UTC m=+3293.214276501"
Oct 30 22:32:31.835919 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:31.835881    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh" event={"ID":"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33","Type":"ContainerStarted","Data":"c3fa0a9489e5af1ddca60646f80f479bf69721af1329409f40b84b67b8f21854"}
Oct 30 22:32:32.747633 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:32.747596    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"
Oct 30 22:32:32.747823 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:32.747794    4871 patch_prober.go:29] interesting pod/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.187:15021/healthz/ready\": dial tcp 10.133.0.187:15021: connect: connection refused" start-of-body=
Oct 30 22:32:32.747876 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:32.747857    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh" podUID="167cb6f4-5f3b-4e79-bd45-3ffc48cefa33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.187:15021/healthz/ready\": dial tcp 10.133.0.187:15021: connect: connection refused"
Oct 30 22:32:33.748206 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:33.748164    4871 patch_prober.go:29] interesting pod/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.187:15021/healthz/ready\": dial tcp 10.133.0.187:15021: connect: connection refused" start-of-body=
Oct 30 22:32:33.748650 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:33.748224    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh" podUID="167cb6f4-5f3b-4e79-bd45-3ffc48cefa33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.187:15021/healthz/ready\": dial tcp 10.133.0.187:15021: connect: connection refused"
Oct 30 22:32:33.818472 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:33.818440    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-355213-6bbf99fb98-l6847"
Oct 30 22:32:34.747375 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:34.747339    4871 patch_prober.go:29] interesting pod/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.187:15021/healthz/ready\": dial tcp 10.133.0.187:15021: connect: connection refused" start-of-body=
Oct 30 22:32:34.747590 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:34.747404    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh" podUID="167cb6f4-5f3b-4e79-bd45-3ffc48cefa33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.187:15021/healthz/ready\": dial tcp 10.133.0.187:15021: connect: connection refused"
Oct 30 22:32:34.932261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:34.931282    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:35.570087 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:32:35.570048    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:32:35.748288 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:35.748210    4871 patch_prober.go:29] interesting pod/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.187:15021/healthz/ready\": dial tcp 10.133.0.187:15021: connect: connection refused" start-of-body=
Oct 30 22:32:35.748470 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:35.748310    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh" podUID="167cb6f4-5f3b-4e79-bd45-3ffc48cefa33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.187:15021/healthz/ready\": dial tcp 10.133.0.187:15021: connect: connection refused"
Oct 30 22:32:36.747776 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:36.747739    4871 patch_prober.go:29] interesting pod/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.187:15021/healthz/ready\": dial tcp 10.133.0.187:15021: connect: connection refused" start-of-body=
Oct 30 22:32:36.748231 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:36.748100    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh" podUID="167cb6f4-5f3b-4e79-bd45-3ffc48cefa33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.187:15021/healthz/ready\": dial tcp 10.133.0.187:15021: connect: connection refused"
Oct 30 22:32:37.753451 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:37.753416    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"
Oct 30 22:32:37.753981 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:37.753897    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"
Oct 30 22:32:37.754832 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:37.754810    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"
Oct 30 22:32:38.650589 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:38.650553    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx_8d401654-8a4c-4a79-b2d3-612765312421/extract/0.log"
Oct 30 22:32:38.652990 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:38.652964    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx_8d401654-8a4c-4a79-b2d3-612765312421/extract/0.log"
Oct 30 22:32:38.663899 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:38.663873    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/2.log"
Oct 30 22:32:38.667175 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:38.667146    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/2.log"
Oct 30 22:32:38.672340 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:38.672317    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzntd_2360fddc-52a5-42a5-82ca-a05aff84d217/ovn-acl-logging/0.log"
Oct 30 22:32:38.674330 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:38.674306    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzntd_2360fddc-52a5-42a5-82ca-a05aff84d217/ovn-acl-logging/0.log"
Oct 30 22:32:39.267061 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:39.267023    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x"]
Oct 30 22:32:39.271388 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:39.271367    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x"
Oct 30 22:32:39.328510 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:39.328467    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x"]
Oct 30 22:32:39.375490 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:39.375443    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w1-323082-serving-cert\""
Oct 30 22:32:39.438499 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:39.438454    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pc9d9\" (UniqueName: \"kubernetes.io/projected/2f8e8b67-0b00-4fb5-b125-865f980b97c0-kube-api-access-pc9d9\") pod \"maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x\" (UID: \"2f8e8b67-0b00-4fb5-b125-865f980b97c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x"
Oct 30 22:32:39.438709 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:39.438519    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f8e8b67-0b00-4fb5-b125-865f980b97c0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x\" (UID: \"2f8e8b67-0b00-4fb5-b125-865f980b97c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x"
Oct 30 22:32:39.438766 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:39.438730    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f8e8b67-0b00-4fb5-b125-865f980b97c0-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x\" (UID: \"2f8e8b67-0b00-4fb5-b125-865f980b97c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x"
Oct 30 22:32:39.438870 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:39.438849    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f8e8b67-0b00-4fb5-b125-865f980b97c0-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x\" (UID: \"2f8e8b67-0b00-4fb5-b125-865f980b97c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x"
Oct 30 22:32:39.466070 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:39.466033    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:39.551311 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:39.547471    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f8e8b67-0b00-4fb5-b125-865f980b97c0-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x\" (UID: \"2f8e8b67-0b00-4fb5-b125-865f980b97c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x"
Oct 30 22:32:39.551311 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:39.547705    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pc9d9\" (UniqueName: \"kubernetes.io/projected/2f8e8b67-0b00-4fb5-b125-865f980b97c0-kube-api-access-pc9d9\") pod \"maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x\" (UID: \"2f8e8b67-0b00-4fb5-b125-865f980b97c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x"
Oct 30 22:32:39.551311 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:39.547752    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f8e8b67-0b00-4fb5-b125-865f980b97c0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x\" (UID: \"2f8e8b67-0b00-4fb5-b125-865f980b97c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x"
Oct 30 22:32:39.551311 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:39.547853    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f8e8b67-0b00-4fb5-b125-865f980b97c0-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x\" (UID: \"2f8e8b67-0b00-4fb5-b125-865f980b97c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x"
Oct 30 22:32:39.551311 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:32:39.548091    4871 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-323082-serving-cert: secret "maas-api-e2e-shared-b-w1-323082-serving-cert" not found
Oct 30 22:32:39.551311 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:32:39.548177    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f8e8b67-0b00-4fb5-b125-865f980b97c0-maas-api-tls podName:2f8e8b67-0b00-4fb5-b125-865f980b97c0 nodeName:}" failed. No retries permitted until 2026-10-30 22:32:40.048153273 +0000 UTC m=+3302.225641887 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2f8e8b67-0b00-4fb5-b125-865f980b97c0-maas-api-tls") pod "maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x" (UID: "2f8e8b67-0b00-4fb5-b125-865f980b97c0") : secret "maas-api-e2e-shared-b-w1-323082-serving-cert" not found
Oct 30 22:32:39.551311 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:39.549518    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f8e8b67-0b00-4fb5-b125-865f980b97c0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x\" (UID: \"2f8e8b67-0b00-4fb5-b125-865f980b97c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x"
Oct 30 22:32:39.571184 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:39.571148    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f8e8b67-0b00-4fb5-b125-865f980b97c0-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x\" (UID: \"2f8e8b67-0b00-4fb5-b125-865f980b97c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x"
Oct 30 22:32:39.617032 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:39.616995    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pc9d9\" (UniqueName: \"kubernetes.io/projected/2f8e8b67-0b00-4fb5-b125-865f980b97c0-kube-api-access-pc9d9\") pod \"maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x\" (UID: \"2f8e8b67-0b00-4fb5-b125-865f980b97c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x"
Oct 30 22:32:39.727467 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:39.727403    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:39.890868 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:39.890823    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:40.062752 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:40.062705    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f8e8b67-0b00-4fb5-b125-865f980b97c0-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x\" (UID: \"2f8e8b67-0b00-4fb5-b125-865f980b97c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x"
Oct 30 22:32:40.065322 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:40.065298    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f8e8b67-0b00-4fb5-b125-865f980b97c0-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x\" (UID: \"2f8e8b67-0b00-4fb5-b125-865f980b97c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x"
Oct 30 22:32:40.186405 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:40.186316    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x"
Oct 30 22:32:40.613461 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:32:40.613425    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f8e8b67_0b00_4fb5_b125_865f980b97c0.slice/crio-df280d0c38b87422b5865fa07aaed94b3f1f031a9efd0bdc6b539922ace0ba5f WatchSource:0}: Error finding container df280d0c38b87422b5865fa07aaed94b3f1f031a9efd0bdc6b539922ace0ba5f: Status 404 returned error can't find the container with id df280d0c38b87422b5865fa07aaed94b3f1f031a9efd0bdc6b539922ace0ba5f
Oct 30 22:32:40.620050 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:40.620014    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x"]
Oct 30 22:32:40.916736 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:40.916685    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x" event={"ID":"2f8e8b67-0b00-4fb5-b125-865f980b97c0","Type":"ContainerStarted","Data":"df280d0c38b87422b5865fa07aaed94b3f1f031a9efd0bdc6b539922ace0ba5f"}
Oct 30 22:32:41.927518 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:41.927469    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x" event={"ID":"2f8e8b67-0b00-4fb5-b125-865f980b97c0","Type":"ContainerStarted","Data":"9ecd4a9578c236b550f6d07365c4905661d1018ba839e6effad0eb9bb43bf35c"}
Oct 30 22:32:41.927904 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:41.927546    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x"
Oct 30 22:32:42.055034 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:42.054976    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x" podStartSLOduration=2.163050556 podStartE2EDuration="3.05495941s" totalImagesPullingTime="891.908854ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:32:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:32:42.051988091 +0000 UTC m=+3304.229476711" watchObservedRunningTime="2026-10-30 22:32:42.05495941 +0000 UTC m=+3304.232448027"
Oct 30 22:32:44.447799 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:44.447764    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:47.577675 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:47.577620    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:47.980820 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:47.980790    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x"
Oct 30 22:32:48.334859 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:48.334707    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:49.033880 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:49.033845    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:49.209976 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:49.209841    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:53.144486 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:53.142313    4871 generic.go:356] "Generic (PLEG): container finished" podID="8acc9cc7-ebae-4d92-8e8c-c8ed8e615030" containerID="3214dd3b7a2df7d71e4ac79864cad8403ab0b9eb2556519fbd6f2d6fe52d17cd" exitCode=0
Oct 30 22:32:53.144486 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:53.142567    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5" event={"ID":"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030","Type":"ContainerDied","Data":"3214dd3b7a2df7d71e4ac79864cad8403ab0b9eb2556519fbd6f2d6fe52d17cd"}
Oct 30 22:32:53.144486 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:53.143085    4871 scope.go:122] "RemoveContainer" containerID="3214dd3b7a2df7d71e4ac79864cad8403ab0b9eb2556519fbd6f2d6fe52d17cd"
Oct 30 22:32:53.147999 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:53.146649    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:32:53.147999 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:53.146748    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:32:53.147999 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:53.146820    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:32:53.809903 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:53.809868    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:54.164262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:54.161745    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5" event={"ID":"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030","Type":"ContainerStarted","Data":"b892b4080ca61e0a9ef2d82e931206ba5e7fb8c28a9bce6d723a67f506ff93d2"}
Oct 30 22:32:54.392961 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:54.392924    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"
Oct 30 22:32:54.393328 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:54.393301    4871 patch_prober.go:29] interesting pod/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.188:15021/healthz/ready\": dial tcp 10.133.0.188:15021: connect: connection refused" start-of-body=
Oct 30 22:32:54.393403 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:54.393370    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5" podUID="8acc9cc7-ebae-4d92-8e8c-c8ed8e615030" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.188:15021/healthz/ready\": dial tcp 10.133.0.188:15021: connect: connection refused"
Oct 30 22:32:55.392790 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:55.392751    4871 patch_prober.go:29] interesting pod/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.188:15021/healthz/ready\": dial tcp 10.133.0.188:15021: connect: connection refused" start-of-body=
Oct 30 22:32:55.393367 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:55.392815    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5" podUID="8acc9cc7-ebae-4d92-8e8c-c8ed8e615030" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.188:15021/healthz/ready\": dial tcp 10.133.0.188:15021: connect: connection refused"
Oct 30 22:32:55.576887 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:55.576843    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:56.393502 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:56.393434    4871 patch_prober.go:29] interesting pod/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.188:15021/healthz/ready\": dial tcp 10.133.0.188:15021: connect: connection refused" start-of-body=
Oct 30 22:32:56.393918 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:56.393536    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5" podUID="8acc9cc7-ebae-4d92-8e8c-c8ed8e615030" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.188:15021/healthz/ready\": dial tcp 10.133.0.188:15021: connect: connection refused"
Oct 30 22:32:57.393247 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:57.393203    4871 patch_prober.go:29] interesting pod/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.188:15021/healthz/ready\": dial tcp 10.133.0.188:15021: connect: connection refused" start-of-body=
Oct 30 22:32:57.393442 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:57.393319    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5" podUID="8acc9cc7-ebae-4d92-8e8c-c8ed8e615030" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.188:15021/healthz/ready\": dial tcp 10.133.0.188:15021: connect: connection refused"
Oct 30 22:32:58.393336 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:58.393294    4871 patch_prober.go:29] interesting pod/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.188:15021/healthz/ready\": dial tcp 10.133.0.188:15021: connect: connection refused" start-of-body=
Oct 30 22:32:58.393879 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:58.393358    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5" podUID="8acc9cc7-ebae-4d92-8e8c-c8ed8e615030" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.188:15021/healthz/ready\": dial tcp 10.133.0.188:15021: connect: connection refused"
Oct 30 22:32:58.964811 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:58.964771    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:59.392893 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:59.392856    4871 patch_prober.go:29] interesting pod/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.188:15021/healthz/ready\": dial tcp 10.133.0.188:15021: connect: connection refused" start-of-body=
Oct 30 22:32:59.393075 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:59.392915    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5" podUID="8acc9cc7-ebae-4d92-8e8c-c8ed8e615030" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.188:15021/healthz/ready\": dial tcp 10.133.0.188:15021: connect: connection refused"
Oct 30 22:32:59.401212 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:59.401182    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:59.890697 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:32:59.890657    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"
Oct 30 22:33:00.399710 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:00.399681    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"
Oct 30 22:33:00.400793 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:00.400766    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"
Oct 30 22:33:03.801226 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:03.801188    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:05.576049 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:33:05.576011    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:33:08.461714 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:08.461679    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:10.648572 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:10.648526    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:13.401057 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:13.401019    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:18.627265 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:18.609971    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:23.090434 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:23.090377    4871 patch_prober.go:29] interesting pod/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.189:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:33:23.090996 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:23.090467    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r" podUID="b873ef79-e292-4496-a9ec-8d2ec09f65f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.189:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:33:23.216369 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:23.216327    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:25.481126 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:25.481028    4871 generic.go:356] "Generic (PLEG): container finished" podID="b873ef79-e292-4496-a9ec-8d2ec09f65f6" containerID="23e5724be2c58f004dd3c8be76117e3f3fa9d18f096a485d1a48c673db63e0f6" exitCode=0
Oct 30 22:33:25.481589 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:25.481107    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r" event={"ID":"b873ef79-e292-4496-a9ec-8d2ec09f65f6","Type":"ContainerDied","Data":"23e5724be2c58f004dd3c8be76117e3f3fa9d18f096a485d1a48c673db63e0f6"}
Oct 30 22:33:25.481747 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:25.481726    4871 scope.go:122] "RemoveContainer" containerID="23e5724be2c58f004dd3c8be76117e3f3fa9d18f096a485d1a48c673db63e0f6"
Oct 30 22:33:25.483601 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:25.483565    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:33:25.483743 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:25.483636    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:33:25.483743 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:25.483673    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:33:26.493992 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:26.493951    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r" event={"ID":"b873ef79-e292-4496-a9ec-8d2ec09f65f6","Type":"ContainerStarted","Data":"09228f534a4ff4ea72dbbc34d77b8564be9aa0a1545a816d176e013d80dbac23"}
Oct 30 22:33:27.031647 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:27.031611    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"
Oct 30 22:33:27.032051 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:27.032018    4871 patch_prober.go:29] interesting pod/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.189:15021/healthz/ready\": dial tcp 10.133.0.189:15021: connect: connection refused" start-of-body=
Oct 30 22:33:27.032332 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:27.032301    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r" podUID="b873ef79-e292-4496-a9ec-8d2ec09f65f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.189:15021/healthz/ready\": dial tcp 10.133.0.189:15021: connect: connection refused"
Oct 30 22:33:27.789698 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:27.789628    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:28.033274 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:28.032684    4871 patch_prober.go:29] interesting pod/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.189:15021/healthz/ready\": dial tcp 10.133.0.189:15021: connect: connection refused" start-of-body=
Oct 30 22:33:28.033274 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:28.032794    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r" podUID="b873ef79-e292-4496-a9ec-8d2ec09f65f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.189:15021/healthz/ready\": dial tcp 10.133.0.189:15021: connect: connection refused"
Oct 30 22:33:29.033262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:29.031674    4871 patch_prober.go:29] interesting pod/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.189:15021/healthz/ready\": dial tcp 10.133.0.189:15021: connect: connection refused" start-of-body=
Oct 30 22:33:29.033262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:29.031740    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r" podUID="b873ef79-e292-4496-a9ec-8d2ec09f65f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.189:15021/healthz/ready\": dial tcp 10.133.0.189:15021: connect: connection refused"
Oct 30 22:33:30.033712 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:30.033671    4871 patch_prober.go:29] interesting pod/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.189:15021/healthz/ready\": dial tcp 10.133.0.189:15021: connect: connection refused" start-of-body=
Oct 30 22:33:30.034292 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:30.034208    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r" podUID="b873ef79-e292-4496-a9ec-8d2ec09f65f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.189:15021/healthz/ready\": dial tcp 10.133.0.189:15021: connect: connection refused"
Oct 30 22:33:31.031885 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:31.031836    4871 patch_prober.go:29] interesting pod/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.189:15021/healthz/ready\": dial tcp 10.133.0.189:15021: connect: connection refused" start-of-body=
Oct 30 22:33:31.032218 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:31.032175    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r" podUID="b873ef79-e292-4496-a9ec-8d2ec09f65f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.189:15021/healthz/ready\": dial tcp 10.133.0.189:15021: connect: connection refused"
Oct 30 22:33:32.039552 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:32.039509    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"
Oct 30 22:33:32.040331 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:32.040305    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"
Oct 30 22:33:32.041657 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:32.041623    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"
Oct 30 22:33:32.543532 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:32.543481    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:35.569866 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:33:35.569817    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:33:37.455964 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:37.455926    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:38.356687 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:38.356654    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:42.446226 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:42.446188    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:44.897861 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:44.897831    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:46.486071 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:46.486033    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:47.542222 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:47.542187    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:47.922506 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:47.922470    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:49.366342 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:49.366308    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:52.073478 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:52.073434    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:56.748027 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:33:56.747992    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:01.600712 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:01.600500    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:02.344652 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:02.344612    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:05.570381 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:34:05.570343    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:34:06.338073 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:06.338022    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:06.487999 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:06.487951    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:10.923608 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:10.923571    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:15.498801 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:15.498763    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:19.651189 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:19.651131    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:20.130494 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:20.130457    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:22.637889 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:22.637846    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"]
Oct 30 22:34:22.638405 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:22.638074    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6" podUID="15fb5463-5e07-43b5-bfb9-f7b1b635c8d6" containerName="istio-proxy" containerID="cri-o://08384472cf5c0b765e58e2e2ed25d17634880c583767b0b21c41f3669ed6ebf5" gracePeriod=30
Oct 30 22:34:24.731370 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:24.731334    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:27.923712 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:27.923643    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"
Oct 30 22:34:27.965380 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:27.965348    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-istiod-ca-cert\") pod \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\" (UID: \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\") "
Oct 30 22:34:27.965571 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:27.965402    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-credential-socket\") pod \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\" (UID: \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\") "
Oct 30 22:34:27.965571 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:27.965432    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-istio-podinfo\") pod \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\" (UID: \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\") "
Oct 30 22:34:27.965571 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:27.965490    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-kube-api-access-brdl4\" (UniqueName: \"kubernetes.io/projected/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-kube-api-access-brdl4\") pod \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\" (UID: \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\") "
Oct 30 22:34:27.965571 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:27.965549    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-workload-certs\") pod \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\" (UID: \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\") "
Oct 30 22:34:27.965571 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:27.965574    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-istio-envoy\") pod \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\" (UID: \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\") "
Oct 30 22:34:27.965883 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:27.965614    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-workload-socket\") pod \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\" (UID: \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\") "
Oct 30 22:34:27.965883 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:27.965702    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-istio-token\" (UniqueName: \"kubernetes.io/projected/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-istio-token\") pod \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\" (UID: \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\") "
Oct 30 22:34:27.965883 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:27.965741    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-istio-data\") pod \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\" (UID: \"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6\") "
Oct 30 22:34:27.965883 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:27.965756    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-credential-socket" pod "15fb5463-5e07-43b5-bfb9-f7b1b635c8d6" (UID: "15fb5463-5e07-43b5-bfb9-f7b1b635c8d6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:27.965883 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:27.965800    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-istiod-ca-cert" pod "15fb5463-5e07-43b5-bfb9-f7b1b635c8d6" (UID: "15fb5463-5e07-43b5-bfb9-f7b1b635c8d6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:34:27.966158 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:27.966105    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-workload-socket" pod "15fb5463-5e07-43b5-bfb9-f7b1b635c8d6" (UID: "15fb5463-5e07-43b5-bfb9-f7b1b635c8d6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:27.966227 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:27.966187    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-istio-data" pod "15fb5463-5e07-43b5-bfb9-f7b1b635c8d6" (UID: "15fb5463-5e07-43b5-bfb9-f7b1b635c8d6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:27.966306 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:27.966279    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-workload-certs" pod "15fb5463-5e07-43b5-bfb9-f7b1b635c8d6" (UID: "15fb5463-5e07-43b5-bfb9-f7b1b635c8d6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:27.966699 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:27.966671    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:27.966833 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:27.966701    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:27.966833 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:27.966717    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:27.966833 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:27.966732    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:27.966833 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:27.966746    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:27.968291 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:27.968261    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-istio-token" pod "15fb5463-5e07-43b5-bfb9-f7b1b635c8d6" (UID: "15fb5463-5e07-43b5-bfb9-f7b1b635c8d6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:34:27.968453 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:27.968435    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-istio-podinfo" pod "15fb5463-5e07-43b5-bfb9-f7b1b635c8d6" (UID: "15fb5463-5e07-43b5-bfb9-f7b1b635c8d6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:34:27.968692 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:27.968662    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-istio-envoy" pod "15fb5463-5e07-43b5-bfb9-f7b1b635c8d6" (UID: "15fb5463-5e07-43b5-bfb9-f7b1b635c8d6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:27.968692 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:27.968683    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-kube-api-access-brdl4" pod "15fb5463-5e07-43b5-bfb9-f7b1b635c8d6" (UID: "15fb5463-5e07-43b5-bfb9-f7b1b635c8d6"). InnerVolumeSpecName "kube-api-access-brdl4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:34:28.069009 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:28.068971    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:28.069009 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:28.069011    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-brdl4\" (UniqueName: \"kubernetes.io/projected/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-kube-api-access-brdl4\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:28.069318 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:28.069029    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:28.069318 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:28.069045    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:28.138205 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:28.138164    4871 generic.go:356] "Generic (PLEG): container finished" podID="15fb5463-5e07-43b5-bfb9-f7b1b635c8d6" containerID="08384472cf5c0b765e58e2e2ed25d17634880c583767b0b21c41f3669ed6ebf5" exitCode=0
Oct 30 22:34:28.138438 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:28.138395    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6" event={"ID":"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6","Type":"ContainerDied","Data":"08384472cf5c0b765e58e2e2ed25d17634880c583767b0b21c41f3669ed6ebf5"}
Oct 30 22:34:28.138519 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:28.138453    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6" event={"ID":"15fb5463-5e07-43b5-bfb9-f7b1b635c8d6","Type":"ContainerDied","Data":"8d013afa14574e53b7de751de20015476651bb818f182e55a39600d6a8a30cf6"}
Oct 30 22:34:28.138519 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:28.138484    4871 scope.go:122] "RemoveContainer" containerID="08384472cf5c0b765e58e2e2ed25d17634880c583767b0b21c41f3669ed6ebf5"
Oct 30 22:34:28.138645 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:28.138637    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"
Oct 30 22:34:28.153451 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:28.153424    4871 scope.go:122] "RemoveContainer" containerID="08384472cf5c0b765e58e2e2ed25d17634880c583767b0b21c41f3669ed6ebf5"
Oct 30 22:34:28.153836 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:34:28.153813    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08384472cf5c0b765e58e2e2ed25d17634880c583767b0b21c41f3669ed6ebf5\": container with ID starting with 08384472cf5c0b765e58e2e2ed25d17634880c583767b0b21c41f3669ed6ebf5 not found: ID does not exist" containerID="08384472cf5c0b765e58e2e2ed25d17634880c583767b0b21c41f3669ed6ebf5"
Oct 30 22:34:28.153915 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:28.153845    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08384472cf5c0b765e58e2e2ed25d17634880c583767b0b21c41f3669ed6ebf5"} err="failed to get container status \"08384472cf5c0b765e58e2e2ed25d17634880c583767b0b21c41f3669ed6ebf5\": rpc error: code = NotFound desc = could not find container \"08384472cf5c0b765e58e2e2ed25d17634880c583767b0b21c41f3669ed6ebf5\": container with ID starting with 08384472cf5c0b765e58e2e2ed25d17634880c583767b0b21c41f3669ed6ebf5 not found: ID does not exist"
Oct 30 22:34:28.323354 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:28.323254    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"]
Oct 30 22:34:28.330082 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:28.330043    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-5dpq6"]
Oct 30 22:34:29.348724 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:29.348684    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:29.962207 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:29.962167    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="15fb5463-5e07-43b5-bfb9-f7b1b635c8d6" path="/var/lib/kubelet/pods/15fb5463-5e07-43b5-bfb9-f7b1b635c8d6/volumes"
Oct 30 22:34:33.915800 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:33.915760    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:35.569488 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:34:35.569440    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:34:37.747089 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:37.747037    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"]
Oct 30 22:34:37.747588 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:37.747390    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n" podUID="40bab067-19f1-4ebc-b467-3f0f2e1bbf48" containerName="istio-proxy" containerID="cri-o://806602b1d25a3284cbc3ceeb0b13abdc9f512f922adc6dbaab6ba3d037f5cbf3" gracePeriod=30
Oct 30 22:34:38.589917 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:38.589870    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:43.057865 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:43.057827    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"
Oct 30 22:34:43.201488 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:43.201435    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-workload-certs\") pod \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\" (UID: \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\") "
Oct 30 22:34:43.201703 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:43.201530    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-istio-token\" (UniqueName: \"kubernetes.io/projected/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-istio-token\") pod \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\" (UID: \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\") "
Oct 30 22:34:43.201703 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:43.201610    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-workload-socket\") pod \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\" (UID: \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\") "
Oct 30 22:34:43.201703 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:43.201631    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-credential-socket\") pod \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\" (UID: \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\") "
Oct 30 22:34:43.201703 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:43.201673    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-kube-api-access-qgxfq\" (UniqueName: \"kubernetes.io/projected/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-kube-api-access-qgxfq\") pod \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\" (UID: \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\") "
Oct 30 22:34:43.201943 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:43.201732    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-istio-podinfo\") pod \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\" (UID: \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\") "
Oct 30 22:34:43.201943 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:43.201761    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-istio-data\") pod \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\" (UID: \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\") "
Oct 30 22:34:43.201943 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:43.201771    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-workload-certs" pod "40bab067-19f1-4ebc-b467-3f0f2e1bbf48" (UID: "40bab067-19f1-4ebc-b467-3f0f2e1bbf48"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:43.201943 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:43.201853    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-istiod-ca-cert\") pod \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\" (UID: \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\") "
Oct 30 22:34:43.201943 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:43.201886    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-istio-envoy\") pod \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\" (UID: \"40bab067-19f1-4ebc-b467-3f0f2e1bbf48\") "
Oct 30 22:34:43.202212 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:43.201934    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-workload-socket" pod "40bab067-19f1-4ebc-b467-3f0f2e1bbf48" (UID: "40bab067-19f1-4ebc-b467-3f0f2e1bbf48"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:43.202212 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:43.202036    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-istio-data" pod "40bab067-19f1-4ebc-b467-3f0f2e1bbf48" (UID: "40bab067-19f1-4ebc-b467-3f0f2e1bbf48"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:43.202212 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:43.202040    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-credential-socket" pod "40bab067-19f1-4ebc-b467-3f0f2e1bbf48" (UID: "40bab067-19f1-4ebc-b467-3f0f2e1bbf48"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:43.202404 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:43.202252    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-istiod-ca-cert" pod "40bab067-19f1-4ebc-b467-3f0f2e1bbf48" (UID: "40bab067-19f1-4ebc-b467-3f0f2e1bbf48"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:34:43.202636 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:43.202571    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:43.202636 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:43.202598    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:43.202636 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:43.202614    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:43.202636 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:43.202628    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:43.202873 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:43.202645    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:43.204116 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:43.204085    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-istio-podinfo" pod "40bab067-19f1-4ebc-b467-3f0f2e1bbf48" (UID: "40bab067-19f1-4ebc-b467-3f0f2e1bbf48"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:34:43.204338 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:43.204146    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-kube-api-access-qgxfq" pod "40bab067-19f1-4ebc-b467-3f0f2e1bbf48" (UID: "40bab067-19f1-4ebc-b467-3f0f2e1bbf48"). InnerVolumeSpecName "kube-api-access-qgxfq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:34:43.204411 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:43.204354    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-istio-token" pod "40bab067-19f1-4ebc-b467-3f0f2e1bbf48" (UID: "40bab067-19f1-4ebc-b467-3f0f2e1bbf48"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:34:43.204592 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:43.204572    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-istio-envoy" pod "40bab067-19f1-4ebc-b467-3f0f2e1bbf48" (UID: "40bab067-19f1-4ebc-b467-3f0f2e1bbf48"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:43.224274 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:43.224204    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:43.303770 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:43.303641    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:43.303770 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:43.303675    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:43.303770 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:43.303686    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgxfq\" (UniqueName: \"kubernetes.io/projected/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-kube-api-access-qgxfq\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:43.303770 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:43.303699    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40bab067-19f1-4ebc-b467-3f0f2e1bbf48-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:43.307231 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:43.307189    4871 generic.go:356] "Generic (PLEG): container finished" podID="40bab067-19f1-4ebc-b467-3f0f2e1bbf48" containerID="806602b1d25a3284cbc3ceeb0b13abdc9f512f922adc6dbaab6ba3d037f5cbf3" exitCode=0
Oct 30 22:34:43.307231 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:43.307251    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"
Oct 30 22:34:43.307559 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:43.307331    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n" event={"ID":"40bab067-19f1-4ebc-b467-3f0f2e1bbf48","Type":"ContainerDied","Data":"806602b1d25a3284cbc3ceeb0b13abdc9f512f922adc6dbaab6ba3d037f5cbf3"}
Oct 30 22:34:43.307559 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:43.307383    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n" event={"ID":"40bab067-19f1-4ebc-b467-3f0f2e1bbf48","Type":"ContainerDied","Data":"a6ca5cf9598297a3e23042b1511e5a9f4d2ab947ad62ed5efce0e52aacd9c8a0"}
Oct 30 22:34:43.307559 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:43.307414    4871 scope.go:122] "RemoveContainer" containerID="806602b1d25a3284cbc3ceeb0b13abdc9f512f922adc6dbaab6ba3d037f5cbf3"
Oct 30 22:34:43.324658 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:43.324628    4871 scope.go:122] "RemoveContainer" containerID="806602b1d25a3284cbc3ceeb0b13abdc9f512f922adc6dbaab6ba3d037f5cbf3"
Oct 30 22:34:43.325087 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:34:43.325050    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"806602b1d25a3284cbc3ceeb0b13abdc9f512f922adc6dbaab6ba3d037f5cbf3\": container with ID starting with 806602b1d25a3284cbc3ceeb0b13abdc9f512f922adc6dbaab6ba3d037f5cbf3 not found: ID does not exist" containerID="806602b1d25a3284cbc3ceeb0b13abdc9f512f922adc6dbaab6ba3d037f5cbf3"
Oct 30 22:34:43.325194 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:43.325101    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"806602b1d25a3284cbc3ceeb0b13abdc9f512f922adc6dbaab6ba3d037f5cbf3"} err="failed to get container status \"806602b1d25a3284cbc3ceeb0b13abdc9f512f922adc6dbaab6ba3d037f5cbf3\": rpc error: code = NotFound desc = could not find container \"806602b1d25a3284cbc3ceeb0b13abdc9f512f922adc6dbaab6ba3d037f5cbf3\": container with ID starting with 806602b1d25a3284cbc3ceeb0b13abdc9f512f922adc6dbaab6ba3d037f5cbf3 not found: ID does not exist"
Oct 30 22:34:43.489062 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:43.489024    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"]
Oct 30 22:34:43.493020 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:43.492992    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-gfc4n"]
Oct 30 22:34:43.962595 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:43.962556    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="40bab067-19f1-4ebc-b467-3f0f2e1bbf48" path="/var/lib/kubelet/pods/40bab067-19f1-4ebc-b467-3f0f2e1bbf48/volumes"
Oct 30 22:34:47.847204 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:47.847163    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:48.313975 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:48.313920    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:49.767597 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:49.767566    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:52.484497 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:52.484462    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:57.142340 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:57.142298    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:57.842393 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:57.842356    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:58.127356 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:58.127318    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"]
Oct 30 22:34:58.179511 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:58.179466    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"]
Oct 30 22:34:58.180047 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:58.179724    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds" podUID="eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11" containerName="istio-proxy" containerID="cri-o://07359f5c86dbdc876784add945ce17fc9546b05db8fcaceb7db7005f886c0c7d" gracePeriod=30
Oct 30 22:34:58.183169 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:58.183142    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"]
Oct 30 22:34:58.183395 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:58.183371    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz" podUID="c647346d-70fe-456e-b81e-f3e90f7bbf17" containerName="istio-proxy" containerID="cri-o://cc5d31fdc84fef9edc42a0f74c93fa929a60a721019709924f398147d30c6acd" gracePeriod=30
Oct 30 22:34:58.440971 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:34:58.440871    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2" podUID="0a69cc06-f8e1-499a-828d-9876d05ed442" containerName="istio-proxy" containerID="cri-o://f145a98c5286f1dd284fada0f395a6a677fa50e96fee6b183b8da091a3a7b5f8" gracePeriod=30
Oct 30 22:35:01.901942 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:01.901905    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:03.497337 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.497299    4871 generic.go:356] "Generic (PLEG): container finished" podID="eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11" containerID="07359f5c86dbdc876784add945ce17fc9546b05db8fcaceb7db7005f886c0c7d" exitCode=0
Oct 30 22:35:03.497956 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.497369    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds" event={"ID":"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11","Type":"ContainerDied","Data":"07359f5c86dbdc876784add945ce17fc9546b05db8fcaceb7db7005f886c0c7d"}
Oct 30 22:35:03.497956 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.497449    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds" event={"ID":"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11","Type":"ContainerDied","Data":"abe1d16ccde714ad43f830e3910e5f78e261a047849c2e9577564feed13eab59"}
Oct 30 22:35:03.497956 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.497475    4871 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="abe1d16ccde714ad43f830e3910e5f78e261a047849c2e9577564feed13eab59"
Oct 30 22:35:03.499791 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.499761    4871 generic.go:356] "Generic (PLEG): container finished" podID="c647346d-70fe-456e-b81e-f3e90f7bbf17" containerID="cc5d31fdc84fef9edc42a0f74c93fa929a60a721019709924f398147d30c6acd" exitCode=0
Oct 30 22:35:03.499939 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.499833    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz" event={"ID":"c647346d-70fe-456e-b81e-f3e90f7bbf17","Type":"ContainerDied","Data":"cc5d31fdc84fef9edc42a0f74c93fa929a60a721019709924f398147d30c6acd"}
Oct 30 22:35:03.582048 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.582018    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"
Oct 30 22:35:03.586404 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.586370    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"
Oct 30 22:35:03.624731 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.624699    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-workload-certs\") pod \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\" (UID: \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\") "
Oct 30 22:35:03.624859 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.624763    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c647346d-70fe-456e-b81e-f3e90f7bbf17-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c647346d-70fe-456e-b81e-f3e90f7bbf17-istio-podinfo\") pod \"c647346d-70fe-456e-b81e-f3e90f7bbf17\" (UID: \"c647346d-70fe-456e-b81e-f3e90f7bbf17\") "
Oct 30 22:35:03.624859 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.624842    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c647346d-70fe-456e-b81e-f3e90f7bbf17-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c647346d-70fe-456e-b81e-f3e90f7bbf17-istio-data\") pod \"c647346d-70fe-456e-b81e-f3e90f7bbf17\" (UID: \"c647346d-70fe-456e-b81e-f3e90f7bbf17\") "
Oct 30 22:35:03.624978 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.624879    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-kube-api-access-xtrjj\" (UniqueName: \"kubernetes.io/projected/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-kube-api-access-xtrjj\") pod \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\" (UID: \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\") "
Oct 30 22:35:03.624978 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.624960    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c647346d-70fe-456e-b81e-f3e90f7bbf17-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c647346d-70fe-456e-b81e-f3e90f7bbf17-istiod-ca-cert\") pod \"c647346d-70fe-456e-b81e-f3e90f7bbf17\" (UID: \"c647346d-70fe-456e-b81e-f3e90f7bbf17\") "
Oct 30 22:35:03.625090 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.625023    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-workload-certs" pod "eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11" (UID: "eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:35:03.625090 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.625041    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-credential-socket\") pod \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\" (UID: \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\") "
Oct 30 22:35:03.625198 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.625088    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-istio-token\" (UniqueName: \"kubernetes.io/projected/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-istio-token\") pod \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\" (UID: \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\") "
Oct 30 22:35:03.625198 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.625115    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-istio-podinfo\") pod \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\" (UID: \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\") "
Oct 30 22:35:03.625337 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.625195    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-istio-data\") pod \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\" (UID: \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\") "
Oct 30 22:35:03.625337 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.625230    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c647346d-70fe-456e-b81e-f3e90f7bbf17-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c647346d-70fe-456e-b81e-f3e90f7bbf17-istio-envoy\") pod \"c647346d-70fe-456e-b81e-f3e90f7bbf17\" (UID: \"c647346d-70fe-456e-b81e-f3e90f7bbf17\") "
Oct 30 22:35:03.625337 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.625290    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c647346d-70fe-456e-b81e-f3e90f7bbf17-istio-data" pod "c647346d-70fe-456e-b81e-f3e90f7bbf17" (UID: "c647346d-70fe-456e-b81e-f3e90f7bbf17"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:35:03.625337 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.625310    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c647346d-70fe-456e-b81e-f3e90f7bbf17-istio-token\" (UniqueName: \"kubernetes.io/projected/c647346d-70fe-456e-b81e-f3e90f7bbf17-istio-token\") pod \"c647346d-70fe-456e-b81e-f3e90f7bbf17\" (UID: \"c647346d-70fe-456e-b81e-f3e90f7bbf17\") "
Oct 30 22:35:03.625621 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.625346    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c647346d-70fe-456e-b81e-f3e90f7bbf17-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c647346d-70fe-456e-b81e-f3e90f7bbf17-workload-certs\") pod \"c647346d-70fe-456e-b81e-f3e90f7bbf17\" (UID: \"c647346d-70fe-456e-b81e-f3e90f7bbf17\") "
Oct 30 22:35:03.625621 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.625372    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-istiod-ca-cert\") pod \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\" (UID: \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\") "
Oct 30 22:35:03.625621 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.625420    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c647346d-70fe-456e-b81e-f3e90f7bbf17-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c647346d-70fe-456e-b81e-f3e90f7bbf17-workload-socket\") pod \"c647346d-70fe-456e-b81e-f3e90f7bbf17\" (UID: \"c647346d-70fe-456e-b81e-f3e90f7bbf17\") "
Oct 30 22:35:03.625621 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.625480    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c647346d-70fe-456e-b81e-f3e90f7bbf17-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c647346d-70fe-456e-b81e-f3e90f7bbf17-credential-socket\") pod \"c647346d-70fe-456e-b81e-f3e90f7bbf17\" (UID: \"c647346d-70fe-456e-b81e-f3e90f7bbf17\") "
Oct 30 22:35:03.625621 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.625508    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-workload-socket\") pod \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\" (UID: \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\") "
Oct 30 22:35:03.625621 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.625552    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c647346d-70fe-456e-b81e-f3e90f7bbf17-kube-api-access-6lk6d\" (UniqueName: \"kubernetes.io/projected/c647346d-70fe-456e-b81e-f3e90f7bbf17-kube-api-access-6lk6d\") pod \"c647346d-70fe-456e-b81e-f3e90f7bbf17\" (UID: \"c647346d-70fe-456e-b81e-f3e90f7bbf17\") "
Oct 30 22:35:03.625621 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.625581    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-istio-envoy\") pod \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\" (UID: \"eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11\") "
Oct 30 22:35:03.626219 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.626199    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:35:03.626295 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.626226    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c647346d-70fe-456e-b81e-f3e90f7bbf17-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:35:03.630404 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.626967    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-credential-socket" pod "eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11" (UID: "eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:35:03.630404 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.628162    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c647346d-70fe-456e-b81e-f3e90f7bbf17-istio-podinfo" pod "c647346d-70fe-456e-b81e-f3e90f7bbf17" (UID: "c647346d-70fe-456e-b81e-f3e90f7bbf17"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:35:03.630404 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.628368    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-workload-socket" pod "eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11" (UID: "eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:35:03.630404 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.628521    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c647346d-70fe-456e-b81e-f3e90f7bbf17-istiod-ca-cert" pod "c647346d-70fe-456e-b81e-f3e90f7bbf17" (UID: "c647346d-70fe-456e-b81e-f3e90f7bbf17"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:35:03.630404 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.629853    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-istiod-ca-cert" pod "eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11" (UID: "eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:35:03.630404 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.629926    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-istio-podinfo" pod "eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11" (UID: "eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:35:03.630404 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.630209    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-istio-data" pod "eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11" (UID: "eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:35:03.631081 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.630478    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c647346d-70fe-456e-b81e-f3e90f7bbf17-workload-certs" pod "c647346d-70fe-456e-b81e-f3e90f7bbf17" (UID: "c647346d-70fe-456e-b81e-f3e90f7bbf17"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:35:03.631081 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.630547    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c647346d-70fe-456e-b81e-f3e90f7bbf17-credential-socket" pod "c647346d-70fe-456e-b81e-f3e90f7bbf17" (UID: "c647346d-70fe-456e-b81e-f3e90f7bbf17"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:35:03.631081 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.630578    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c647346d-70fe-456e-b81e-f3e90f7bbf17-workload-socket" pod "c647346d-70fe-456e-b81e-f3e90f7bbf17" (UID: "c647346d-70fe-456e-b81e-f3e90f7bbf17"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:35:03.633773 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.633730    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-istio-envoy" pod "eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11" (UID: "eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:35:03.634014 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.633981    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c647346d-70fe-456e-b81e-f3e90f7bbf17-istio-envoy" pod "c647346d-70fe-456e-b81e-f3e90f7bbf17" (UID: "c647346d-70fe-456e-b81e-f3e90f7bbf17"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:35:03.634610 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.634574    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-istio-token" pod "eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11" (UID: "eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:35:03.636111 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.636078    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c647346d-70fe-456e-b81e-f3e90f7bbf17-istio-token" pod "c647346d-70fe-456e-b81e-f3e90f7bbf17" (UID: "c647346d-70fe-456e-b81e-f3e90f7bbf17"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:35:03.637047 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.637009    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c647346d-70fe-456e-b81e-f3e90f7bbf17-kube-api-access-6lk6d" pod "c647346d-70fe-456e-b81e-f3e90f7bbf17" (UID: "c647346d-70fe-456e-b81e-f3e90f7bbf17"). InnerVolumeSpecName "kube-api-access-6lk6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:35:03.637813 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.637779    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-kube-api-access-xtrjj" pod "eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11" (UID: "eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11"). InnerVolumeSpecName "kube-api-access-xtrjj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:35:03.711359 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.711329    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"
Oct 30 22:35:03.727157 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.727093    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a69cc06-f8e1-499a-828d-9876d05ed442-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a69cc06-f8e1-499a-828d-9876d05ed442-credential-socket\") pod \"0a69cc06-f8e1-499a-828d-9876d05ed442\" (UID: \"0a69cc06-f8e1-499a-828d-9876d05ed442\") "
Oct 30 22:35:03.727157 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.727125    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a69cc06-f8e1-499a-828d-9876d05ed442-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a69cc06-f8e1-499a-828d-9876d05ed442-istio-envoy\") pod \"0a69cc06-f8e1-499a-828d-9876d05ed442\" (UID: \"0a69cc06-f8e1-499a-828d-9876d05ed442\") "
Oct 30 22:35:03.727410 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.727176    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a69cc06-f8e1-499a-828d-9876d05ed442-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a69cc06-f8e1-499a-828d-9876d05ed442-istio-data\") pod \"0a69cc06-f8e1-499a-828d-9876d05ed442\" (UID: \"0a69cc06-f8e1-499a-828d-9876d05ed442\") "
Oct 30 22:35:03.727410 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.727206    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a69cc06-f8e1-499a-828d-9876d05ed442-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a69cc06-f8e1-499a-828d-9876d05ed442-workload-certs\") pod \"0a69cc06-f8e1-499a-828d-9876d05ed442\" (UID: \"0a69cc06-f8e1-499a-828d-9876d05ed442\") "
Oct 30 22:35:03.727410 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.727268    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0a69cc06-f8e1-499a-828d-9876d05ed442-kube-api-access-8x9l9\" (UniqueName: \"kubernetes.io/projected/0a69cc06-f8e1-499a-828d-9876d05ed442-kube-api-access-8x9l9\") pod \"0a69cc06-f8e1-499a-828d-9876d05ed442\" (UID: \"0a69cc06-f8e1-499a-828d-9876d05ed442\") "
Oct 30 22:35:03.727410 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.727323    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0a69cc06-f8e1-499a-828d-9876d05ed442-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a69cc06-f8e1-499a-828d-9876d05ed442-istio-podinfo\") pod \"0a69cc06-f8e1-499a-828d-9876d05ed442\" (UID: \"0a69cc06-f8e1-499a-828d-9876d05ed442\") "
Oct 30 22:35:03.727410 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.727365    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a69cc06-f8e1-499a-828d-9876d05ed442-credential-socket" pod "0a69cc06-f8e1-499a-828d-9876d05ed442" (UID: "0a69cc06-f8e1-499a-828d-9876d05ed442"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:35:03.727410 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.727383    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a69cc06-f8e1-499a-828d-9876d05ed442-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a69cc06-f8e1-499a-828d-9876d05ed442-workload-socket\") pod \"0a69cc06-f8e1-499a-828d-9876d05ed442\" (UID: \"0a69cc06-f8e1-499a-828d-9876d05ed442\") "
Oct 30 22:35:03.727756 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.727491    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0a69cc06-f8e1-499a-828d-9876d05ed442-istio-token\" (UniqueName: \"kubernetes.io/projected/0a69cc06-f8e1-499a-828d-9876d05ed442-istio-token\") pod \"0a69cc06-f8e1-499a-828d-9876d05ed442\" (UID: \"0a69cc06-f8e1-499a-828d-9876d05ed442\") "
Oct 30 22:35:03.727756 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.727523    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0a69cc06-f8e1-499a-828d-9876d05ed442-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a69cc06-f8e1-499a-828d-9876d05ed442-istiod-ca-cert\") pod \"0a69cc06-f8e1-499a-828d-9876d05ed442\" (UID: \"0a69cc06-f8e1-499a-828d-9876d05ed442\") "
Oct 30 22:35:03.727756 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.727526    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a69cc06-f8e1-499a-828d-9876d05ed442-workload-certs" pod "0a69cc06-f8e1-499a-828d-9876d05ed442" (UID: "0a69cc06-f8e1-499a-828d-9876d05ed442"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:35:03.727756 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.727533    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a69cc06-f8e1-499a-828d-9876d05ed442-istio-data" pod "0a69cc06-f8e1-499a-828d-9876d05ed442" (UID: "0a69cc06-f8e1-499a-828d-9876d05ed442"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:35:03.728035 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.727762    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a69cc06-f8e1-499a-828d-9876d05ed442-workload-socket" pod "0a69cc06-f8e1-499a-828d-9876d05ed442" (UID: "0a69cc06-f8e1-499a-828d-9876d05ed442"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:35:03.728035 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.727857    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a69cc06-f8e1-499a-828d-9876d05ed442-istiod-ca-cert" pod "0a69cc06-f8e1-499a-828d-9876d05ed442" (UID: "0a69cc06-f8e1-499a-828d-9876d05ed442"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:35:03.728265 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.728208    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c647346d-70fe-456e-b81e-f3e90f7bbf17-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:35:03.728265 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.728254    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c647346d-70fe-456e-b81e-f3e90f7bbf17-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:35:03.728412 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.728271    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:35:03.728412 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.728285    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c647346d-70fe-456e-b81e-f3e90f7bbf17-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:35:03.728412 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.728303    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c647346d-70fe-456e-b81e-f3e90f7bbf17-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:35:03.728412 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.728318    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:35:03.728412 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.728334    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6lk6d\" (UniqueName: \"kubernetes.io/projected/c647346d-70fe-456e-b81e-f3e90f7bbf17-kube-api-access-6lk6d\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:35:03.728412 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.728351    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a69cc06-f8e1-499a-828d-9876d05ed442-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:35:03.728412 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.728367    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:35:03.728412 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.728381    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c647346d-70fe-456e-b81e-f3e90f7bbf17-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:35:03.728412 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.728396    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a69cc06-f8e1-499a-828d-9876d05ed442-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:35:03.728412 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.728411    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a69cc06-f8e1-499a-828d-9876d05ed442-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:35:03.728986 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.728425    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtrjj\" (UniqueName: \"kubernetes.io/projected/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-kube-api-access-xtrjj\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:35:03.728986 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.728440    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c647346d-70fe-456e-b81e-f3e90f7bbf17-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:35:03.728986 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.728456    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:35:03.728986 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.728472    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a69cc06-f8e1-499a-828d-9876d05ed442-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:35:03.728986 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.728487    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:35:03.728986 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.728502    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:35:03.728986 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.728518    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:35:03.728986 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.728532    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c647346d-70fe-456e-b81e-f3e90f7bbf17-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:35:03.728986 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.728547    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a69cc06-f8e1-499a-828d-9876d05ed442-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:35:03.729560 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.729534    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0a69cc06-f8e1-499a-828d-9876d05ed442-istio-podinfo" pod "0a69cc06-f8e1-499a-828d-9876d05ed442" (UID: "0a69cc06-f8e1-499a-828d-9876d05ed442"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:35:03.729666 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.729579    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a69cc06-f8e1-499a-828d-9876d05ed442-kube-api-access-8x9l9" pod "0a69cc06-f8e1-499a-828d-9876d05ed442" (UID: "0a69cc06-f8e1-499a-828d-9876d05ed442"). InnerVolumeSpecName "kube-api-access-8x9l9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:35:03.729852 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.729834    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a69cc06-f8e1-499a-828d-9876d05ed442-istio-envoy" pod "0a69cc06-f8e1-499a-828d-9876d05ed442" (UID: "0a69cc06-f8e1-499a-828d-9876d05ed442"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:35:03.729852 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.729842    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a69cc06-f8e1-499a-828d-9876d05ed442-istio-token" pod "0a69cc06-f8e1-499a-828d-9876d05ed442" (UID: "0a69cc06-f8e1-499a-828d-9876d05ed442"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:35:03.829847 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.829808    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a69cc06-f8e1-499a-828d-9876d05ed442-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:35:03.829847 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.829842    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8x9l9\" (UniqueName: \"kubernetes.io/projected/0a69cc06-f8e1-499a-828d-9876d05ed442-kube-api-access-8x9l9\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:35:03.829847 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.829855    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a69cc06-f8e1-499a-828d-9876d05ed442-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:35:03.830109 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:03.829864    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a69cc06-f8e1-499a-828d-9876d05ed442-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:35:04.510432 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:04.510340    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"
Oct 30 22:35:04.510926 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:04.510349    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz" event={"ID":"c647346d-70fe-456e-b81e-f3e90f7bbf17","Type":"ContainerDied","Data":"053e88db4d5e894336d879ffe93a8658fd0a60b286a32e8add60f30ffd1eeff2"}
Oct 30 22:35:04.510926 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:04.510513    4871 scope.go:122] "RemoveContainer" containerID="cc5d31fdc84fef9edc42a0f74c93fa929a60a721019709924f398147d30c6acd"
Oct 30 22:35:04.512221 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:04.512189    4871 generic.go:356] "Generic (PLEG): container finished" podID="0a69cc06-f8e1-499a-828d-9876d05ed442" containerID="f145a98c5286f1dd284fada0f395a6a677fa50e96fee6b183b8da091a3a7b5f8" exitCode=0
Oct 30 22:35:04.512372 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:04.512286    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2" event={"ID":"0a69cc06-f8e1-499a-828d-9876d05ed442","Type":"ContainerDied","Data":"f145a98c5286f1dd284fada0f395a6a677fa50e96fee6b183b8da091a3a7b5f8"}
Oct 30 22:35:04.512372 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:04.512333    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"
Oct 30 22:35:04.512372 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:04.512362    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"
Oct 30 22:35:04.512539 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:04.512360    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2" event={"ID":"0a69cc06-f8e1-499a-828d-9876d05ed442","Type":"ContainerDied","Data":"4ac8064ca4906ee48c28a46d381950d22058e2a6363b944254b33a20b3c4d1de"}
Oct 30 22:35:04.522386 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:04.522363    4871 scope.go:122] "RemoveContainer" containerID="f145a98c5286f1dd284fada0f395a6a677fa50e96fee6b183b8da091a3a7b5f8"
Oct 30 22:35:04.533100 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:04.533076    4871 scope.go:122] "RemoveContainer" containerID="f145a98c5286f1dd284fada0f395a6a677fa50e96fee6b183b8da091a3a7b5f8"
Oct 30 22:35:04.533471 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:35:04.533449    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f145a98c5286f1dd284fada0f395a6a677fa50e96fee6b183b8da091a3a7b5f8\": container with ID starting with f145a98c5286f1dd284fada0f395a6a677fa50e96fee6b183b8da091a3a7b5f8 not found: ID does not exist" containerID="f145a98c5286f1dd284fada0f395a6a677fa50e96fee6b183b8da091a3a7b5f8"
Oct 30 22:35:04.533536 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:04.533483    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f145a98c5286f1dd284fada0f395a6a677fa50e96fee6b183b8da091a3a7b5f8"} err="failed to get container status \"f145a98c5286f1dd284fada0f395a6a677fa50e96fee6b183b8da091a3a7b5f8\": rpc error: code = NotFound desc = could not find container \"f145a98c5286f1dd284fada0f395a6a677fa50e96fee6b183b8da091a3a7b5f8\": container with ID starting with f145a98c5286f1dd284fada0f395a6a677fa50e96fee6b183b8da091a3a7b5f8 not found: ID does not exist"
Oct 30 22:35:04.685472 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:04.685420    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"]
Oct 30 22:35:04.737517 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:04.737483    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-cn6j2"]
Oct 30 22:35:04.805197 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:04.805092    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"]
Oct 30 22:35:04.812633 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:04.812600    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-ph4rz"]
Oct 30 22:35:04.931279 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:04.931226    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"]
Oct 30 22:35:04.935014 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:04.934986    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-26fds"]
Oct 30 22:35:05.569344 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:35:05.569302    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:35:05.909227 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:05.909184    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w1-ef00a4/auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s"]
Oct 30 22:35:05.910163 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:05.910133    4871 cpu_manager.go:414] "containerMap: removing container" podUID="15fb5463-5e07-43b5-bfb9-f7b1b635c8d6" containerName="istio-proxy"
Oct 30 22:35:05.910163 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:05.910162    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="15fb5463-5e07-43b5-bfb9-f7b1b635c8d6" containerName="istio-proxy"
Oct 30 22:35:05.910368 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:05.910197    4871 cpu_manager.go:414] "containerMap: removing container" podUID="0a69cc06-f8e1-499a-828d-9876d05ed442" containerName="istio-proxy"
Oct 30 22:35:05.910368 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:05.910206    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0a69cc06-f8e1-499a-828d-9876d05ed442" containerName="istio-proxy"
Oct 30 22:35:05.910368 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:05.910230    4871 cpu_manager.go:414] "containerMap: removing container" podUID="eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11" containerName="istio-proxy"
Oct 30 22:35:05.910368 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:05.910253    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11" containerName="istio-proxy"
Oct 30 22:35:05.910368 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:05.910280    4871 cpu_manager.go:414] "containerMap: removing container" podUID="40bab067-19f1-4ebc-b467-3f0f2e1bbf48" containerName="istio-proxy"
Oct 30 22:35:05.910368 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:05.910287    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40bab067-19f1-4ebc-b467-3f0f2e1bbf48" containerName="istio-proxy"
Oct 30 22:35:05.910368 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:05.910310    4871 cpu_manager.go:414] "containerMap: removing container" podUID="c647346d-70fe-456e-b81e-f3e90f7bbf17" containerName="istio-proxy"
Oct 30 22:35:05.910368 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:05.910318    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c647346d-70fe-456e-b81e-f3e90f7bbf17" containerName="istio-proxy"
Oct 30 22:35:05.910778 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:05.910469    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="40bab067-19f1-4ebc-b467-3f0f2e1bbf48" containerName="istio-proxy"
Oct 30 22:35:05.910778 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:05.910485    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="c647346d-70fe-456e-b81e-f3e90f7bbf17" containerName="istio-proxy"
Oct 30 22:35:05.910778 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:05.910502    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="15fb5463-5e07-43b5-bfb9-f7b1b635c8d6" containerName="istio-proxy"
Oct 30 22:35:05.910778 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:05.910512    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="0a69cc06-f8e1-499a-828d-9876d05ed442" containerName="istio-proxy"
Oct 30 22:35:05.910778 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:05.910526    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11" containerName="istio-proxy"
Oct 30 22:35:05.926403 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:05.926211    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w1-ef00a4/auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s"
Oct 30 22:35:05.956373 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:05.956337    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6832135c-e3d3-4fca-bc16-0a83a6f6a798-model-cache\") pod \"auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s\" (UID: \"6832135c-e3d3-4fca-bc16-0a83a6f6a798\") " pod="ai-tenant-e2e-shared-a-w1-ef00a4/auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s"
Oct 30 22:35:05.956590 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:05.956386    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6832135c-e3d3-4fca-bc16-0a83a6f6a798-dshm\") pod \"auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s\" (UID: \"6832135c-e3d3-4fca-bc16-0a83a6f6a798\") " pod="ai-tenant-e2e-shared-a-w1-ef00a4/auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s"
Oct 30 22:35:05.956590 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:05.956478    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6832135c-e3d3-4fca-bc16-0a83a6f6a798-tls-certs\") pod \"auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s\" (UID: \"6832135c-e3d3-4fca-bc16-0a83a6f6a798\") " pod="ai-tenant-e2e-shared-a-w1-ef00a4/auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s"
Oct 30 22:35:05.956590 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:05.956544    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjb2q\" (UniqueName: \"kubernetes.io/projected/6832135c-e3d3-4fca-bc16-0a83a6f6a798-kube-api-access-zjb2q\") pod \"auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s\" (UID: \"6832135c-e3d3-4fca-bc16-0a83a6f6a798\") " pod="ai-tenant-e2e-shared-a-w1-ef00a4/auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s"
Oct 30 22:35:05.956761 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:05.956623    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6832135c-e3d3-4fca-bc16-0a83a6f6a798-tmp-dir\") pod \"auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s\" (UID: \"6832135c-e3d3-4fca-bc16-0a83a6f6a798\") " pod="ai-tenant-e2e-shared-a-w1-ef00a4/auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s"
Oct 30 22:35:05.956761 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:05.956654    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6832135c-e3d3-4fca-bc16-0a83a6f6a798-home\") pod \"auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s\" (UID: \"6832135c-e3d3-4fca-bc16-0a83a6f6a798\") " pod="ai-tenant-e2e-shared-a-w1-ef00a4/auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s"
Oct 30 22:35:05.966867 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:05.966830    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0a69cc06-f8e1-499a-828d-9876d05ed442" path="/var/lib/kubelet/pods/0a69cc06-f8e1-499a-828d-9876d05ed442/volumes"
Oct 30 22:35:05.967465 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:05.967446    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c647346d-70fe-456e-b81e-f3e90f7bbf17" path="/var/lib/kubelet/pods/c647346d-70fe-456e-b81e-f3e90f7bbf17/volumes"
Oct 30 22:35:05.967912 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:05.967897    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11" path="/var/lib/kubelet/pods/eb9ceb3c-0af0-4522-8b0a-8ba8d67ecb11/volumes"
Oct 30 22:35:05.968586 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:05.968569    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w1-ef00a4/auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s"]
Oct 30 22:35:06.001910 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:06.001873    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:06.015101 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:06.015063    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w1-ef00a4\"/\"default-dockercfg-gdsrn\""
Oct 30 22:35:06.015309 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:06.015128    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w1-ef00a4\"/\"auth-test-model-ef00a4-kserve-self-signed-certs\""
Oct 30 22:35:06.015309 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:06.015174    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w1-ef00a4\"/\"kube-root-ca.crt\""
Oct 30 22:35:06.034305 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:06.034275    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w1-ef00a4\"/\"openshift-service-ca.crt\""
Oct 30 22:35:06.069263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:06.063064    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6832135c-e3d3-4fca-bc16-0a83a6f6a798-tls-certs\") pod \"auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s\" (UID: \"6832135c-e3d3-4fca-bc16-0a83a6f6a798\") " pod="ai-tenant-e2e-shared-a-w1-ef00a4/auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s"
Oct 30 22:35:06.069263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:06.063213    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjb2q\" (UniqueName: \"kubernetes.io/projected/6832135c-e3d3-4fca-bc16-0a83a6f6a798-kube-api-access-zjb2q\") pod \"auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s\" (UID: \"6832135c-e3d3-4fca-bc16-0a83a6f6a798\") " pod="ai-tenant-e2e-shared-a-w1-ef00a4/auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s"
Oct 30 22:35:06.078562 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:06.071339    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6832135c-e3d3-4fca-bc16-0a83a6f6a798-tmp-dir\") pod \"auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s\" (UID: \"6832135c-e3d3-4fca-bc16-0a83a6f6a798\") " pod="ai-tenant-e2e-shared-a-w1-ef00a4/auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s"
Oct 30 22:35:06.078562 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:06.071401    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6832135c-e3d3-4fca-bc16-0a83a6f6a798-home\") pod \"auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s\" (UID: \"6832135c-e3d3-4fca-bc16-0a83a6f6a798\") " pod="ai-tenant-e2e-shared-a-w1-ef00a4/auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s"
Oct 30 22:35:06.078562 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:06.071587    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6832135c-e3d3-4fca-bc16-0a83a6f6a798-model-cache\") pod \"auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s\" (UID: \"6832135c-e3d3-4fca-bc16-0a83a6f6a798\") " pod="ai-tenant-e2e-shared-a-w1-ef00a4/auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s"
Oct 30 22:35:06.078562 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:06.071641    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6832135c-e3d3-4fca-bc16-0a83a6f6a798-dshm\") pod \"auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s\" (UID: \"6832135c-e3d3-4fca-bc16-0a83a6f6a798\") " pod="ai-tenant-e2e-shared-a-w1-ef00a4/auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s"
Oct 30 22:35:06.097109 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:06.089134    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6832135c-e3d3-4fca-bc16-0a83a6f6a798-tmp-dir\") pod \"auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s\" (UID: \"6832135c-e3d3-4fca-bc16-0a83a6f6a798\") " pod="ai-tenant-e2e-shared-a-w1-ef00a4/auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s"
Oct 30 22:35:06.097109 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:06.089443    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6832135c-e3d3-4fca-bc16-0a83a6f6a798-home\") pod \"auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s\" (UID: \"6832135c-e3d3-4fca-bc16-0a83a6f6a798\") " pod="ai-tenant-e2e-shared-a-w1-ef00a4/auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s"
Oct 30 22:35:06.097109 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:06.089935    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6832135c-e3d3-4fca-bc16-0a83a6f6a798-model-cache\") pod \"auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s\" (UID: \"6832135c-e3d3-4fca-bc16-0a83a6f6a798\") " pod="ai-tenant-e2e-shared-a-w1-ef00a4/auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s"
Oct 30 22:35:06.104960 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:06.104923    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6832135c-e3d3-4fca-bc16-0a83a6f6a798-dshm\") pod \"auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s\" (UID: \"6832135c-e3d3-4fca-bc16-0a83a6f6a798\") " pod="ai-tenant-e2e-shared-a-w1-ef00a4/auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s"
Oct 30 22:35:06.105499 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:06.105302    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6832135c-e3d3-4fca-bc16-0a83a6f6a798-tls-certs\") pod \"auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s\" (UID: \"6832135c-e3d3-4fca-bc16-0a83a6f6a798\") " pod="ai-tenant-e2e-shared-a-w1-ef00a4/auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s"
Oct 30 22:35:06.151256 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:06.151211    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjb2q\" (UniqueName: \"kubernetes.io/projected/6832135c-e3d3-4fca-bc16-0a83a6f6a798-kube-api-access-zjb2q\") pod \"auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s\" (UID: \"6832135c-e3d3-4fca-bc16-0a83a6f6a798\") " pod="ai-tenant-e2e-shared-a-w1-ef00a4/auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s"
Oct 30 22:35:06.262453 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:06.262360    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w1-ef00a4/auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s"
Oct 30 22:35:06.491691 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:06.491545    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:06.663785 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:35:06.663747    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6832135c_e3d3_4fca_bc16_0a83a6f6a798.slice/crio-73f23078cd31428c3aa199c7c4285d87a28e0d7fe0e871ea346921587e00395f WatchSource:0}: Error finding container 73f23078cd31428c3aa199c7c4285d87a28e0d7fe0e871ea346921587e00395f: Status 404 returned error can't find the container with id 73f23078cd31428c3aa199c7c4285d87a28e0d7fe0e871ea346921587e00395f
Oct 30 22:35:06.668714 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:06.668689    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w1-ef00a4/auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s"]
Oct 30 22:35:07.564294 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:07.564229    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w1-ef00a4/auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s" event={"ID":"6832135c-e3d3-4fca-bc16-0a83a6f6a798","Type":"ContainerStarted","Data":"9c6ba627ba15fbaa1d88e90aeb81d6551e9b795f95bdacf573c349e862481b18"}
Oct 30 22:35:07.564759 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:07.564311    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w1-ef00a4/auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s" event={"ID":"6832135c-e3d3-4fca-bc16-0a83a6f6a798","Type":"ContainerStarted","Data":"73f23078cd31428c3aa199c7c4285d87a28e0d7fe0e871ea346921587e00395f"}
Oct 30 22:35:07.690331 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:07.690266    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w1-ef00a4/auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s" podStartSLOduration=2.690250219 podStartE2EDuration="2.690250219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:35:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:35:07.686891774 +0000 UTC m=+3449.864380402" watchObservedRunningTime="2026-10-30 22:35:07.690250219 +0000 UTC m=+3449.867738827"
Oct 30 22:35:08.263002 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:08.262963    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w1-ef00a4/auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s"
Oct 30 22:35:11.018381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:11.018345    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:15.555279 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:15.555232    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:16.263337 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:16.263300    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w1-ef00a4/auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s"
Oct 30 22:35:16.276303 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:16.276272    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w1-ef00a4/auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s"
Oct 30 22:35:16.651872 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:16.651841    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w1-ef00a4/auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s"
Oct 30 22:35:20.132299 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:20.132214    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:24.720720 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:24.720685    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:25.406712 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:25.406664    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:29.305430 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:29.305366    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:33.915046 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:33.915010    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:35.569591 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:35:35.569549    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:35:38.806267 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:38.806045    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:41.043480 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:41.043445    4871 scope.go:122] "RemoveContainer" containerID="07359f5c86dbdc876784add945ce17fc9546b05db8fcaceb7db7005f886c0c7d"
Oct 30 22:35:41.899427 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:41.899370    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:43.448863 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:43.448830    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:47.587303 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:47.587267    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:47.951937 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:47.951902    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:48.321223 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:48.321137    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:48.472960 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:48.472928    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:48.698459 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:48.698423    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:50.105123 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:50.105082    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:50.390016 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:50.389970    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:52.666627 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:52.666591    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:56.401070 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:56.401033    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:56.522396 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:56.522360    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-355213-6bbf99fb98-l6847"]
Oct 30 22:35:56.522660 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:56.522628    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-355213-6bbf99fb98-l6847" podUID="b78eced2-5755-49b5-9666-3af5f738c437" containerName="maas-api" containerID="cri-o://dbc4ab4e324853931dbaefd112d936b6e948797777866dc81399a01634b9def2" gracePeriod=30
Oct 30 22:35:56.619072 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:56.619026    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:56.798286 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:56.796462    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:56.850430 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:56.850401    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-355213-6bbf99fb98-l6847"
Oct 30 22:35:56.962132 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:56.962032    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b78eced2-5755-49b5-9666-3af5f738c437-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b78eced2-5755-49b5-9666-3af5f738c437-maas-api-tls\") pod \"b78eced2-5755-49b5-9666-3af5f738c437\" (UID: \"b78eced2-5755-49b5-9666-3af5f738c437\") "
Oct 30 22:35:56.962377 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:56.962359    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b78eced2-5755-49b5-9666-3af5f738c437-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b78eced2-5755-49b5-9666-3af5f738c437-openshift-service-ca\") pod \"b78eced2-5755-49b5-9666-3af5f738c437\" (UID: \"b78eced2-5755-49b5-9666-3af5f738c437\") "
Oct 30 22:35:56.962626 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:56.962614    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b78eced2-5755-49b5-9666-3af5f738c437-kube-api-access-cbfhp\" (UniqueName: \"kubernetes.io/projected/b78eced2-5755-49b5-9666-3af5f738c437-kube-api-access-cbfhp\") pod \"b78eced2-5755-49b5-9666-3af5f738c437\" (UID: \"b78eced2-5755-49b5-9666-3af5f738c437\") "
Oct 30 22:35:56.962752 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:56.962741    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b78eced2-5755-49b5-9666-3af5f738c437-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b78eced2-5755-49b5-9666-3af5f738c437-metrics-certs\") pod \"b78eced2-5755-49b5-9666-3af5f738c437\" (UID: \"b78eced2-5755-49b5-9666-3af5f738c437\") "
Oct 30 22:35:56.966081 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:56.966047    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b78eced2-5755-49b5-9666-3af5f738c437-openshift-service-ca" pod "b78eced2-5755-49b5-9666-3af5f738c437" (UID: "b78eced2-5755-49b5-9666-3af5f738c437"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:35:56.966871 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:56.966837    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b78eced2-5755-49b5-9666-3af5f738c437-maas-api-tls" pod "b78eced2-5755-49b5-9666-3af5f738c437" (UID: "b78eced2-5755-49b5-9666-3af5f738c437"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:35:56.967130 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:56.967111    4871 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b78eced2-5755-49b5-9666-3af5f738c437-openshift-service-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:35:56.973453 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:56.973410    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b78eced2-5755-49b5-9666-3af5f738c437-kube-api-access-cbfhp" pod "b78eced2-5755-49b5-9666-3af5f738c437" (UID: "b78eced2-5755-49b5-9666-3af5f738c437"). InnerVolumeSpecName "kube-api-access-cbfhp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:35:56.980855 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:56.980820    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b78eced2-5755-49b5-9666-3af5f738c437-metrics-certs" pod "b78eced2-5755-49b5-9666-3af5f738c437" (UID: "b78eced2-5755-49b5-9666-3af5f738c437"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:35:57.006284 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:57.004972    4871 generic.go:356] "Generic (PLEG): container finished" podID="b78eced2-5755-49b5-9666-3af5f738c437" containerID="dbc4ab4e324853931dbaefd112d936b6e948797777866dc81399a01634b9def2" exitCode=0
Oct 30 22:35:57.006284 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:57.005060    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-355213-6bbf99fb98-l6847"
Oct 30 22:35:57.006284 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:57.005187    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-355213-6bbf99fb98-l6847" event={"ID":"b78eced2-5755-49b5-9666-3af5f738c437","Type":"ContainerDied","Data":"dbc4ab4e324853931dbaefd112d936b6e948797777866dc81399a01634b9def2"}
Oct 30 22:35:57.006284 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:57.005273    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-355213-6bbf99fb98-l6847" event={"ID":"b78eced2-5755-49b5-9666-3af5f738c437","Type":"ContainerDied","Data":"045994d17792941366e0b74b899ae88a94015ff2713ecfd67e0aa0c582fbe94c"}
Oct 30 22:35:57.006284 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:57.005305    4871 scope.go:122] "RemoveContainer" containerID="dbc4ab4e324853931dbaefd112d936b6e948797777866dc81399a01634b9def2"
Oct 30 22:35:57.034436 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:57.034402    4871 scope.go:122] "RemoveContainer" containerID="dbc4ab4e324853931dbaefd112d936b6e948797777866dc81399a01634b9def2"
Oct 30 22:35:57.034934 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:35:57.034904    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbc4ab4e324853931dbaefd112d936b6e948797777866dc81399a01634b9def2\": container with ID starting with dbc4ab4e324853931dbaefd112d936b6e948797777866dc81399a01634b9def2 not found: ID does not exist" containerID="dbc4ab4e324853931dbaefd112d936b6e948797777866dc81399a01634b9def2"
Oct 30 22:35:57.035052 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:57.034945    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbc4ab4e324853931dbaefd112d936b6e948797777866dc81399a01634b9def2"} err="failed to get container status \"dbc4ab4e324853931dbaefd112d936b6e948797777866dc81399a01634b9def2\": rpc error: code = NotFound desc = could not find container \"dbc4ab4e324853931dbaefd112d936b6e948797777866dc81399a01634b9def2\": container with ID starting with dbc4ab4e324853931dbaefd112d936b6e948797777866dc81399a01634b9def2 not found: ID does not exist"
Oct 30 22:35:57.069063 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:57.069025    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbfhp\" (UniqueName: \"kubernetes.io/projected/b78eced2-5755-49b5-9666-3af5f738c437-kube-api-access-cbfhp\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:35:57.069063 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:57.069056    4871 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b78eced2-5755-49b5-9666-3af5f738c437-metrics-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:35:57.069063 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:57.069070    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b78eced2-5755-49b5-9666-3af5f738c437-maas-api-tls\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:35:57.179433 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:57.179400    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-355213-6bbf99fb98-l6847"]
Oct 30 22:35:57.186201 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:57.186168    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-355213-6bbf99fb98-l6847"]
Oct 30 22:35:57.962506 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:57.962471    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b78eced2-5755-49b5-9666-3af5f738c437" path="/var/lib/kubelet/pods/b78eced2-5755-49b5-9666-3af5f738c437/volumes"
Oct 30 22:35:58.087816 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:58.087764    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:58.312735 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:58.312643    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:58.512927 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:58.512893    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:59.128897 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:35:59.128863    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:03.407719 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:03.407639    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:05.569892 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:36:05.569847    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:36:05.891448 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:05.891410    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:06.116052 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:06.116020    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:06.286295 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:06.286197    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:06.900321 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:06.900232    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:07.582268 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:07.579167    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:07.816260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:07.814841    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:08.285616 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:08.285582    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:09.088697 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:09.088664    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:09.358254 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:09.358160    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:09.533326 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:09.533285    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:13.046512 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:13.046481    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:15.139411 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:15.139374    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"]
Oct 30 22:36:15.140161 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:15.140132    4871 cpu_manager.go:414] "containerMap: removing container" podUID="b78eced2-5755-49b5-9666-3af5f738c437" containerName="maas-api"
Oct 30 22:36:15.140161 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:15.140155    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b78eced2-5755-49b5-9666-3af5f738c437" containerName="maas-api"
Oct 30 22:36:15.140415 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:15.140268    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="b78eced2-5755-49b5-9666-3af5f738c437" containerName="maas-api"
Oct 30 22:36:15.143968 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:15.143942    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"
Oct 30 22:36:15.199258 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:15.199201    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-5bb0b1-openshift-default-dockercfg-ckxgl\""
Oct 30 22:36:15.259599 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:15.259560    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e96f69ad-3e18-4188-9dad-67d37845519a-credential-socket\") pod \"e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt\" (UID: \"e96f69ad-3e18-4188-9dad-67d37845519a\") " pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"
Oct 30 22:36:15.259909 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:15.259885    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e96f69ad-3e18-4188-9dad-67d37845519a-istio-data\") pod \"e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt\" (UID: \"e96f69ad-3e18-4188-9dad-67d37845519a\") " pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"
Oct 30 22:36:15.260110 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:15.260092    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e96f69ad-3e18-4188-9dad-67d37845519a-workload-socket\") pod \"e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt\" (UID: \"e96f69ad-3e18-4188-9dad-67d37845519a\") " pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"
Oct 30 22:36:15.260634 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:15.260579    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e96f69ad-3e18-4188-9dad-67d37845519a-istio-token\") pod \"e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt\" (UID: \"e96f69ad-3e18-4188-9dad-67d37845519a\") " pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"
Oct 30 22:36:15.260817 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:15.260797    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e96f69ad-3e18-4188-9dad-67d37845519a-istiod-ca-cert\") pod \"e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt\" (UID: \"e96f69ad-3e18-4188-9dad-67d37845519a\") " pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"
Oct 30 22:36:15.261156 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:15.261136    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e96f69ad-3e18-4188-9dad-67d37845519a-istio-podinfo\") pod \"e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt\" (UID: \"e96f69ad-3e18-4188-9dad-67d37845519a\") " pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"
Oct 30 22:36:15.261346 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:15.261319    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdxt8\" (UniqueName: \"kubernetes.io/projected/e96f69ad-3e18-4188-9dad-67d37845519a-kube-api-access-tdxt8\") pod \"e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt\" (UID: \"e96f69ad-3e18-4188-9dad-67d37845519a\") " pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"
Oct 30 22:36:15.261509 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:15.261391    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e96f69ad-3e18-4188-9dad-67d37845519a-istio-envoy\") pod \"e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt\" (UID: \"e96f69ad-3e18-4188-9dad-67d37845519a\") " pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"
Oct 30 22:36:15.261509 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:15.261469    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e96f69ad-3e18-4188-9dad-67d37845519a-workload-certs\") pod \"e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt\" (UID: \"e96f69ad-3e18-4188-9dad-67d37845519a\") " pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"
Oct 30 22:36:15.262984 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:15.262960    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"]
Oct 30 22:36:15.363304 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:15.363229    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e96f69ad-3e18-4188-9dad-67d37845519a-workload-socket\") pod \"e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt\" (UID: \"e96f69ad-3e18-4188-9dad-67d37845519a\") " pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"
Oct 30 22:36:15.363511 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:15.363355    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e96f69ad-3e18-4188-9dad-67d37845519a-istio-token\") pod \"e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt\" (UID: \"e96f69ad-3e18-4188-9dad-67d37845519a\") " pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"
Oct 30 22:36:15.363511 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:15.363388    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e96f69ad-3e18-4188-9dad-67d37845519a-istiod-ca-cert\") pod \"e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt\" (UID: \"e96f69ad-3e18-4188-9dad-67d37845519a\") " pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"
Oct 30 22:36:15.363511 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:15.363416    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e96f69ad-3e18-4188-9dad-67d37845519a-istio-podinfo\") pod \"e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt\" (UID: \"e96f69ad-3e18-4188-9dad-67d37845519a\") " pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"
Oct 30 22:36:15.363642 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:15.363535    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdxt8\" (UniqueName: \"kubernetes.io/projected/e96f69ad-3e18-4188-9dad-67d37845519a-kube-api-access-tdxt8\") pod \"e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt\" (UID: \"e96f69ad-3e18-4188-9dad-67d37845519a\") " pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"
Oct 30 22:36:15.363642 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:15.363609    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e96f69ad-3e18-4188-9dad-67d37845519a-istio-envoy\") pod \"e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt\" (UID: \"e96f69ad-3e18-4188-9dad-67d37845519a\") " pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"
Oct 30 22:36:15.363727 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:15.363674    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e96f69ad-3e18-4188-9dad-67d37845519a-workload-certs\") pod \"e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt\" (UID: \"e96f69ad-3e18-4188-9dad-67d37845519a\") " pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"
Oct 30 22:36:15.363727 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:15.363705    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e96f69ad-3e18-4188-9dad-67d37845519a-credential-socket\") pod \"e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt\" (UID: \"e96f69ad-3e18-4188-9dad-67d37845519a\") " pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"
Oct 30 22:36:15.363727 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:15.363716    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e96f69ad-3e18-4188-9dad-67d37845519a-workload-socket\") pod \"e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt\" (UID: \"e96f69ad-3e18-4188-9dad-67d37845519a\") " pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"
Oct 30 22:36:15.363864 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:15.363781    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e96f69ad-3e18-4188-9dad-67d37845519a-istio-data\") pod \"e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt\" (UID: \"e96f69ad-3e18-4188-9dad-67d37845519a\") " pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"
Oct 30 22:36:15.364053 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:15.364031    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e96f69ad-3e18-4188-9dad-67d37845519a-credential-socket\") pod \"e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt\" (UID: \"e96f69ad-3e18-4188-9dad-67d37845519a\") " pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"
Oct 30 22:36:15.364224 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:15.364103    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e96f69ad-3e18-4188-9dad-67d37845519a-istio-data\") pod \"e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt\" (UID: \"e96f69ad-3e18-4188-9dad-67d37845519a\") " pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"
Oct 30 22:36:15.364224 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:15.364148    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e96f69ad-3e18-4188-9dad-67d37845519a-istiod-ca-cert\") pod \"e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt\" (UID: \"e96f69ad-3e18-4188-9dad-67d37845519a\") " pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"
Oct 30 22:36:15.364224 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:15.364159    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e96f69ad-3e18-4188-9dad-67d37845519a-workload-certs\") pod \"e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt\" (UID: \"e96f69ad-3e18-4188-9dad-67d37845519a\") " pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"
Oct 30 22:36:15.365822 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:15.365793    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e96f69ad-3e18-4188-9dad-67d37845519a-istio-envoy\") pod \"e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt\" (UID: \"e96f69ad-3e18-4188-9dad-67d37845519a\") " pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"
Oct 30 22:36:15.366059 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:15.366030    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e96f69ad-3e18-4188-9dad-67d37845519a-istio-podinfo\") pod \"e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt\" (UID: \"e96f69ad-3e18-4188-9dad-67d37845519a\") " pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"
Oct 30 22:36:15.428193 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:15.428118    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdxt8\" (UniqueName: \"kubernetes.io/projected/e96f69ad-3e18-4188-9dad-67d37845519a-kube-api-access-tdxt8\") pod \"e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt\" (UID: \"e96f69ad-3e18-4188-9dad-67d37845519a\") " pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"
Oct 30 22:36:15.428587 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:15.428566    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e96f69ad-3e18-4188-9dad-67d37845519a-istio-token\") pod \"e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt\" (UID: \"e96f69ad-3e18-4188-9dad-67d37845519a\") " pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"
Oct 30 22:36:15.459596 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:15.459559    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"
Oct 30 22:36:15.888632 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:36:15.888588    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode96f69ad_3e18_4188_9dad_67d37845519a.slice/crio-c4ad688138744bb664ebc42ab2a4101bdaafc87fdd2a49a9afc14c3efbcbba6c WatchSource:0}: Error finding container c4ad688138744bb664ebc42ab2a4101bdaafc87fdd2a49a9afc14c3efbcbba6c: Status 404 returned error can't find the container with id c4ad688138744bb664ebc42ab2a4101bdaafc87fdd2a49a9afc14c3efbcbba6c
Oct 30 22:36:15.891377 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:15.891338    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:36:15.891493 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:15.891429    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:36:15.891493 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:15.891475    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:36:15.893196 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:15.893167    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"]
Oct 30 22:36:16.169915 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:16.169814    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt" event={"ID":"e96f69ad-3e18-4188-9dad-67d37845519a","Type":"ContainerStarted","Data":"6636d970fa4893812ae6961d9720883bcdedc9561cd288ff6cafae9a81dc4153"}
Oct 30 22:36:16.169915 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:16.169882    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt" event={"ID":"e96f69ad-3e18-4188-9dad-67d37845519a","Type":"ContainerStarted","Data":"c4ad688138744bb664ebc42ab2a4101bdaafc87fdd2a49a9afc14c3efbcbba6c"}
Oct 30 22:36:16.296139 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:16.296087    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt" podStartSLOduration=1.296071821 podStartE2EDuration="1.296071821s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:36:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:36:16.295257964 +0000 UTC m=+3518.472746577" watchObservedRunningTime="2026-10-30 22:36:16.296071821 +0000 UTC m=+3518.473560438"
Oct 30 22:36:16.460487 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:16.460398    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"
Oct 30 22:36:16.462437 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:16.462413    4871 patch_prober.go:29] interesting pod/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.194:15021/healthz/ready\": dial tcp 10.133.0.194:15021: connect: connection refused" start-of-body=
Oct 30 22:36:16.462567 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:16.462461    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt" podUID="e96f69ad-3e18-4188-9dad-67d37845519a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.194:15021/healthz/ready\": dial tcp 10.133.0.194:15021: connect: connection refused"
Oct 30 22:36:17.460663 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:17.460624    4871 patch_prober.go:29] interesting pod/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.194:15021/healthz/ready\": dial tcp 10.133.0.194:15021: connect: connection refused" start-of-body=
Oct 30 22:36:17.461067 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:17.460681    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt" podUID="e96f69ad-3e18-4188-9dad-67d37845519a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.194:15021/healthz/ready\": dial tcp 10.133.0.194:15021: connect: connection refused"
Oct 30 22:36:17.609209 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:17.609138    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:18.460514 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:18.460473    4871 patch_prober.go:29] interesting pod/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.194:15021/healthz/ready\": dial tcp 10.133.0.194:15021: connect: connection refused" start-of-body=
Oct 30 22:36:18.460685 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:18.460544    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt" podUID="e96f69ad-3e18-4188-9dad-67d37845519a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.194:15021/healthz/ready\": dial tcp 10.133.0.194:15021: connect: connection refused"
Oct 30 22:36:19.460176 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:19.460138    4871 patch_prober.go:29] interesting pod/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.194:15021/healthz/ready\": dial tcp 10.133.0.194:15021: connect: connection refused" start-of-body=
Oct 30 22:36:19.460641 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:19.460204    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt" podUID="e96f69ad-3e18-4188-9dad-67d37845519a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.194:15021/healthz/ready\": dial tcp 10.133.0.194:15021: connect: connection refused"
Oct 30 22:36:20.460144 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:20.460106    4871 patch_prober.go:29] interesting pod/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.194:15021/healthz/ready\": dial tcp 10.133.0.194:15021: connect: connection refused" start-of-body=
Oct 30 22:36:20.460406 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:20.460193    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt" podUID="e96f69ad-3e18-4188-9dad-67d37845519a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.194:15021/healthz/ready\": dial tcp 10.133.0.194:15021: connect: connection refused"
Oct 30 22:36:21.460694 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:21.460651    4871 patch_prober.go:29] interesting pod/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.194:15021/healthz/ready\": dial tcp 10.133.0.194:15021: connect: connection refused" start-of-body=
Oct 30 22:36:21.461206 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:21.460712    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt" podUID="e96f69ad-3e18-4188-9dad-67d37845519a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.194:15021/healthz/ready\": dial tcp 10.133.0.194:15021: connect: connection refused"
Oct 30 22:36:22.283635 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:22.283598    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:22.464995 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:22.464959    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"
Oct 30 22:36:22.465411 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:22.465247    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"
Oct 30 22:36:22.465973 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:22.465954    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"
Oct 30 22:36:27.289597 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:27.289564    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:31.606897 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:31.606841    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:35.570269 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:36:35.570219    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:36:36.246983 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:36.246923    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:38.630259 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:38.630196    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"]
Oct 30 22:36:38.630857 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:38.630414    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb" podUID="e631c7a2-bd8e-4a9f-8768-6fa43cde6080" containerName="istio-proxy" containerID="cri-o://74b8bf64a7107c96a9243e2797b6593aa4c62bef354a2db709b66dedbb741d00" gracePeriod=30
Oct 30 22:36:40.878915 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:40.878880    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:43.930185 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:43.930152    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"
Oct 30 22:36:43.979567 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:43.979530    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-istio-data\") pod \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\" (UID: \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\") "
Oct 30 22:36:43.979759 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:43.979582    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-workload-socket\") pod \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\" (UID: \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\") "
Oct 30 22:36:43.979759 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:43.979662    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-istio-envoy\") pod \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\" (UID: \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\") "
Oct 30 22:36:43.979759 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:43.979723    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-kube-api-access-x6j2f\" (UniqueName: \"kubernetes.io/projected/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-kube-api-access-x6j2f\") pod \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\" (UID: \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\") "
Oct 30 22:36:43.979953 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:43.979763    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-istio-token\" (UniqueName: \"kubernetes.io/projected/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-istio-token\") pod \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\" (UID: \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\") "
Oct 30 22:36:43.979953 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:43.979820    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-credential-socket\") pod \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\" (UID: \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\") "
Oct 30 22:36:43.979953 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:43.979846    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-istio-data" pod "e631c7a2-bd8e-4a9f-8768-6fa43cde6080" (UID: "e631c7a2-bd8e-4a9f-8768-6fa43cde6080"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:36:43.979953 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:43.979898    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-workload-certs\") pod \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\" (UID: \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\") "
Oct 30 22:36:43.979953 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:43.979905    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-workload-socket" pod "e631c7a2-bd8e-4a9f-8768-6fa43cde6080" (UID: "e631c7a2-bd8e-4a9f-8768-6fa43cde6080"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:36:43.980253 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:43.980151    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-workload-certs" pod "e631c7a2-bd8e-4a9f-8768-6fa43cde6080" (UID: "e631c7a2-bd8e-4a9f-8768-6fa43cde6080"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:36:43.980364 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:43.980344    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-credential-socket" pod "e631c7a2-bd8e-4a9f-8768-6fa43cde6080" (UID: "e631c7a2-bd8e-4a9f-8768-6fa43cde6080"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:36:43.980661 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:43.980635    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-istio-podinfo\") pod \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\" (UID: \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\") "
Oct 30 22:36:43.980767 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:43.980751    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-istiod-ca-cert\") pod \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\" (UID: \"e631c7a2-bd8e-4a9f-8768-6fa43cde6080\") "
Oct 30 22:36:43.981344 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:43.981319    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:36:43.981464 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:43.981347    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:36:43.981464 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:43.981375    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:36:43.981464 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:43.981389    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:36:43.981464 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:43.981395    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-istiod-ca-cert" pod "e631c7a2-bd8e-4a9f-8768-6fa43cde6080" (UID: "e631c7a2-bd8e-4a9f-8768-6fa43cde6080"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:36:43.982600 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:43.982574    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-istio-token" pod "e631c7a2-bd8e-4a9f-8768-6fa43cde6080" (UID: "e631c7a2-bd8e-4a9f-8768-6fa43cde6080"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:36:43.983497 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:43.983471    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-istio-envoy" pod "e631c7a2-bd8e-4a9f-8768-6fa43cde6080" (UID: "e631c7a2-bd8e-4a9f-8768-6fa43cde6080"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:36:43.983879 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:43.983854    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-kube-api-access-x6j2f" pod "e631c7a2-bd8e-4a9f-8768-6fa43cde6080" (UID: "e631c7a2-bd8e-4a9f-8768-6fa43cde6080"). InnerVolumeSpecName "kube-api-access-x6j2f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:36:43.984367 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:43.984331    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-istio-podinfo" pod "e631c7a2-bd8e-4a9f-8768-6fa43cde6080" (UID: "e631c7a2-bd8e-4a9f-8768-6fa43cde6080"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:36:44.082950 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:44.082898    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:36:44.082950 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:44.082947    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:36:44.083230 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:44.082963    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:36:44.083230 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:44.082978    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6j2f\" (UniqueName: \"kubernetes.io/projected/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-kube-api-access-x6j2f\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:36:44.083230 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:44.082994    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e631c7a2-bd8e-4a9f-8768-6fa43cde6080-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:36:44.432186 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:44.432151    4871 generic.go:356] "Generic (PLEG): container finished" podID="e631c7a2-bd8e-4a9f-8768-6fa43cde6080" containerID="74b8bf64a7107c96a9243e2797b6593aa4c62bef354a2db709b66dedbb741d00" exitCode=0
Oct 30 22:36:44.432405 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:44.432200    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"
Oct 30 22:36:44.432405 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:44.432266    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb" event={"ID":"e631c7a2-bd8e-4a9f-8768-6fa43cde6080","Type":"ContainerDied","Data":"74b8bf64a7107c96a9243e2797b6593aa4c62bef354a2db709b66dedbb741d00"}
Oct 30 22:36:44.432405 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:44.432326    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb" event={"ID":"e631c7a2-bd8e-4a9f-8768-6fa43cde6080","Type":"ContainerDied","Data":"9442901bbb9e2ef0a764802aae175d2bb32ddc239ecd566e3a2e369b0875c5cb"}
Oct 30 22:36:44.432405 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:44.432350    4871 scope.go:122] "RemoveContainer" containerID="74b8bf64a7107c96a9243e2797b6593aa4c62bef354a2db709b66dedbb741d00"
Oct 30 22:36:44.444172 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:44.444152    4871 scope.go:122] "RemoveContainer" containerID="74b8bf64a7107c96a9243e2797b6593aa4c62bef354a2db709b66dedbb741d00"
Oct 30 22:36:44.444471 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:36:44.444445    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74b8bf64a7107c96a9243e2797b6593aa4c62bef354a2db709b66dedbb741d00\": container with ID starting with 74b8bf64a7107c96a9243e2797b6593aa4c62bef354a2db709b66dedbb741d00 not found: ID does not exist" containerID="74b8bf64a7107c96a9243e2797b6593aa4c62bef354a2db709b66dedbb741d00"
Oct 30 22:36:44.444565 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:44.444477    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74b8bf64a7107c96a9243e2797b6593aa4c62bef354a2db709b66dedbb741d00"} err="failed to get container status \"74b8bf64a7107c96a9243e2797b6593aa4c62bef354a2db709b66dedbb741d00\": rpc error: code = NotFound desc = could not find container \"74b8bf64a7107c96a9243e2797b6593aa4c62bef354a2db709b66dedbb741d00\": container with ID starting with 74b8bf64a7107c96a9243e2797b6593aa4c62bef354a2db709b66dedbb741d00 not found: ID does not exist"
Oct 30 22:36:44.609403 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:44.609362    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"]
Oct 30 22:36:44.659302 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:44.659263    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb"]
Oct 30 22:36:44.819106 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:44.819016    4871 patch_prober.go:29] interesting pod/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.174:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:36:44.819321 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:44.819103    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-56f3fe-openshift-default-784cd69b87-tqqzb" podUID="e631c7a2-bd8e-4a9f-8768-6fa43cde6080" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.174:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:36:45.508871 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:45.508839    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:45.962154 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:45.962120    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e631c7a2-bd8e-4a9f-8768-6fa43cde6080" path="/var/lib/kubelet/pods/e631c7a2-bd8e-4a9f-8768-6fa43cde6080/volumes"
Oct 30 22:36:49.037553 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:49.037448    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:49.296729 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:49.296633    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:50.064569 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:50.064499    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:50.467931 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:50.467895    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:54.591445 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:54.591392    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:59.137195 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:36:59.137129    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:03.779462 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:37:03.779427    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:05.055514 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:37:05.055476    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:05.568191 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:37:05.568155    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:37:08.459848 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:37:08.459764    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:12.937362 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:37:12.937324    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:13.090711 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:37:13.090662    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:17.664517 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:37:17.664476    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:22.158762 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:37:22.158711    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:25.101680 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:37:25.101644    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:26.804513 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:37:26.804479    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:31.388405 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:37:31.388367    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:35.570517 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:37:35.570477    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:37:36.019199 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:37:36.019164    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:38.717072 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:37:38.717038    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx_8d401654-8a4c-4a79-b2d3-612765312421/extract/0.log"
Oct 30 22:37:38.721014 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:37:38.720992    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx_8d401654-8a4c-4a79-b2d3-612765312421/extract/0.log"
Oct 30 22:37:38.730118 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:37:38.730089    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/2.log"
Oct 30 22:37:38.734083 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:37:38.734056    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/2.log"
Oct 30 22:37:38.738095 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:37:38.738072    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzntd_2360fddc-52a5-42a5-82ca-a05aff84d217/ovn-acl-logging/0.log"
Oct 30 22:37:38.742061 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:37:38.742039    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzntd_2360fddc-52a5-42a5-82ca-a05aff84d217/ovn-acl-logging/0.log"
Oct 30 22:37:40.620392 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:37:40.620360    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:45.302000 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:37:45.301969    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:48.340918 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:37:48.340876    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:48.496438 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:37:48.496404    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:49.378140 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:37:49.378096    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:49.822757 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:37:49.822667    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:50.596021 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:37:50.595977    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:50.796492 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:37:50.796449    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:50.960263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:37:50.957831    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:51.125708 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:37:51.125671    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:51.287339 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:37:51.287231    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:53.204566 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:37:53.204528    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:54.461299 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:37:54.461256    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:56.296322 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:37:56.296270    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:59.086329 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:37:59.084606    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:03.556011 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:38:03.555973    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:05.569362 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:38:05.569323    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:38:08.132166 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:38:08.132123    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:12.726919 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:38:12.726864    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:17.414959 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:38:17.414912    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:22.014292 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:38:22.014166    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:26.618878 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:38:26.618835    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:31.242278 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:38:31.242222    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:33.381916 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:38:33.381880    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:35.572069 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:38:35.572028    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:38:35.804789 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:38:35.804745    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:40.358304 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:38:40.358265    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:43.937357 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:38:43.937319    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:44.650877 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:38:44.650839    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:44.829652 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:38:44.829618    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:45.030269 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:38:45.030168    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:49.336673 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:38:49.336617    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:49.517288 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:38:49.517230    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:49.671888 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:38:49.671858    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:49.831593 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:38:49.831544    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:51.133016 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:38:51.132975    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:52.679481 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:38:52.679419    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:52.866958 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:38:52.866867    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:54.627719 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:38:54.627668    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:59.185215 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:38:59.185169    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:03.740798 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:39:03.740763    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:05.571025 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:39:05.570975    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:39:08.342640 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:39:08.342592    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:12.928759 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:39:12.928723    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:17.604481 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:39:17.604442    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:22.216598 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:39:22.216564    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:26.797125 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:39:26.797086    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:31.342644 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:39:31.342611    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:35.569709 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:39:35.569670    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:39:35.898667 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:39:35.898633    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:40.498044 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:39:40.498005    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:45.041873 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:39:45.041821    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:49.591474 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:39:49.591397    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:50.115362 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:39:50.115323    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:50.338299 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:39:50.338264    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:51.362782 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:39:51.362741    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:51.522371 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:39:51.522336    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:54.156929 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:39:54.156890    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:54.314964 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:39:54.314927    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:54.476459 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:39:54.476383    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:58.918153 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:39:58.918103    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:03.629727 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:40:03.629683    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:05.570355 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:40:05.570320    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:40:08.232993 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:40:08.232923    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:12.831936 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:40:12.831899    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:17.186705 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:40:17.186641    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:17.337944 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:40:17.337908    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:21.926184 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:40:21.926143    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:26.622401 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:40:26.622358    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:29.408831 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:40:29.408798    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:30.935687 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:40:30.935638    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:31.327496 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:40:31.327410    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:35.571581 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:40:35.571541    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:40:35.905301 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:40:35.905265    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:40.523814 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:40:40.523778    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:45.107115 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:40:45.107074    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:49.829918 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:40:49.829878    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:50.471910 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:40:50.471868    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:51.829711 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:40:51.829655    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:54.335736 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:40:54.335680    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:54.519697 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:40:54.519659    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:55.210322 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:40:55.210289    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:55.763926 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:40:55.763889    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:56.218326 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:40:56.218287    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:56.379713 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:40:56.379677    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:59.078110 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:40:59.078068    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:59.238510 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:40:59.238470    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:59.421274 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:40:59.421220    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:02.451264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:41:02.451194    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:02.636049 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:41:02.636011    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:04.199843 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:41:04.199808    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:05.569840 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:41:05.569802    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:41:07.251589 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:41:07.251553    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:08.886190 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:41:08.886146    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:12.762312 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:41:12.762268    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:13.311545 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:41:13.311502    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:13.773877 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:41:13.773835    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:13.953252 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:41:13.953188    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:14.119724 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:41:14.119686    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:16.240302 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:41:16.240265    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:18.123927 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:41:18.123807    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:18.325074 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:41:18.325031    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:18.475817 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:41:18.475713    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:18.723993 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:41:18.723965    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:23.772009 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:41:23.771968    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:24.858014 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:41:24.857976    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:25.069311 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:41:25.055827    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:27.872224 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:41:27.872189    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:28.580690 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:41:28.580626    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:28.760420 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:41:28.760380    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:28.918573 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:41:28.918540    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:32.229145 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:41:32.229109    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:32.426035 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:41:32.425998    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:33.608152 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:41:33.608093    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:35.422205 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:41:35.422164    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:35.612251 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:41:35.612059    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:35.663350 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:41:35.663273    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:41:38.706686 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:41:38.706651    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:43.419416 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:41:43.419358    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:48.011156 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:41:48.011118    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:50.826891 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:41:50.826854    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:52.228347 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:41:52.228310    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:52.597397 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:41:52.597301    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:57.137246 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:41:57.137194    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:58.657109 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:41:58.657066    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w1-ef00a4/auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s"]
Oct 30 22:41:58.680380 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:41:58.680346    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:01.703033 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:01.702995    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:05.571961 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:42:05.571918    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:42:06.237685 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:06.237649    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:10.861471 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:10.861436    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:11.018264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:11.018220    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:11.244691 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:11.244657    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:13.678582 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:13.678516    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w1-ef00a4/auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s" podUID="6832135c-e3d3-4fca-bc16-0a83a6f6a798" containerName="main" containerID="cri-o://9c6ba627ba15fbaa1d88e90aeb81d6551e9b795f95bdacf573c349e862481b18" gracePeriod=45
Oct 30 22:42:14.209806 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:14.209763    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w1-ef00a4/auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s"
Oct 30 22:42:14.296146 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:14.296058    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6832135c-e3d3-4fca-bc16-0a83a6f6a798-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6832135c-e3d3-4fca-bc16-0a83a6f6a798-model-cache\") pod \"6832135c-e3d3-4fca-bc16-0a83a6f6a798\" (UID: \"6832135c-e3d3-4fca-bc16-0a83a6f6a798\") "
Oct 30 22:42:14.296357 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:14.296173    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6832135c-e3d3-4fca-bc16-0a83a6f6a798-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6832135c-e3d3-4fca-bc16-0a83a6f6a798-tmp-dir\") pod \"6832135c-e3d3-4fca-bc16-0a83a6f6a798\" (UID: \"6832135c-e3d3-4fca-bc16-0a83a6f6a798\") "
Oct 30 22:42:14.296357 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:14.296287    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6832135c-e3d3-4fca-bc16-0a83a6f6a798-kube-api-access-zjb2q\" (UniqueName: \"kubernetes.io/projected/6832135c-e3d3-4fca-bc16-0a83a6f6a798-kube-api-access-zjb2q\") pod \"6832135c-e3d3-4fca-bc16-0a83a6f6a798\" (UID: \"6832135c-e3d3-4fca-bc16-0a83a6f6a798\") "
Oct 30 22:42:14.296357 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:14.296346    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6832135c-e3d3-4fca-bc16-0a83a6f6a798-dshm\" (UniqueName: \"kubernetes.io/empty-dir/6832135c-e3d3-4fca-bc16-0a83a6f6a798-dshm\") pod \"6832135c-e3d3-4fca-bc16-0a83a6f6a798\" (UID: \"6832135c-e3d3-4fca-bc16-0a83a6f6a798\") "
Oct 30 22:42:14.296488 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:14.296376    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6832135c-e3d3-4fca-bc16-0a83a6f6a798-home\" (UniqueName: \"kubernetes.io/empty-dir/6832135c-e3d3-4fca-bc16-0a83a6f6a798-home\") pod \"6832135c-e3d3-4fca-bc16-0a83a6f6a798\" (UID: \"6832135c-e3d3-4fca-bc16-0a83a6f6a798\") "
Oct 30 22:42:14.296488 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:14.296403    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6832135c-e3d3-4fca-bc16-0a83a6f6a798-model-cache" pod "6832135c-e3d3-4fca-bc16-0a83a6f6a798" (UID: "6832135c-e3d3-4fca-bc16-0a83a6f6a798"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:42:14.296488 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:14.296411    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6832135c-e3d3-4fca-bc16-0a83a6f6a798-tmp-dir" pod "6832135c-e3d3-4fca-bc16-0a83a6f6a798" (UID: "6832135c-e3d3-4fca-bc16-0a83a6f6a798"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:42:14.296721 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:14.296695    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6832135c-e3d3-4fca-bc16-0a83a6f6a798-home" pod "6832135c-e3d3-4fca-bc16-0a83a6f6a798" (UID: "6832135c-e3d3-4fca-bc16-0a83a6f6a798"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:42:14.296839 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:14.296820    4871 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6832135c-e3d3-4fca-bc16-0a83a6f6a798-tmp-dir\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:42:14.296905 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:14.296841    4871 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6832135c-e3d3-4fca-bc16-0a83a6f6a798-home\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:42:14.296905 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:14.296850    4871 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6832135c-e3d3-4fca-bc16-0a83a6f6a798-model-cache\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:42:14.298631 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:14.298608    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6832135c-e3d3-4fca-bc16-0a83a6f6a798-dshm" pod "6832135c-e3d3-4fca-bc16-0a83a6f6a798" (UID: "6832135c-e3d3-4fca-bc16-0a83a6f6a798"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:42:14.298773 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:14.298758    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6832135c-e3d3-4fca-bc16-0a83a6f6a798-kube-api-access-zjb2q" pod "6832135c-e3d3-4fca-bc16-0a83a6f6a798" (UID: "6832135c-e3d3-4fca-bc16-0a83a6f6a798"). InnerVolumeSpecName "kube-api-access-zjb2q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:42:14.397620 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:14.397572    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6832135c-e3d3-4fca-bc16-0a83a6f6a798-tls-certs\" (UniqueName: \"kubernetes.io/secret/6832135c-e3d3-4fca-bc16-0a83a6f6a798-tls-certs\") pod \"6832135c-e3d3-4fca-bc16-0a83a6f6a798\" (UID: \"6832135c-e3d3-4fca-bc16-0a83a6f6a798\") "
Oct 30 22:42:14.398007 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:14.397984    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjb2q\" (UniqueName: \"kubernetes.io/projected/6832135c-e3d3-4fca-bc16-0a83a6f6a798-kube-api-access-zjb2q\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:42:14.398091 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:14.398012    4871 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6832135c-e3d3-4fca-bc16-0a83a6f6a798-dshm\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:42:14.399700 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:14.399666    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6832135c-e3d3-4fca-bc16-0a83a6f6a798-tls-certs" pod "6832135c-e3d3-4fca-bc16-0a83a6f6a798" (UID: "6832135c-e3d3-4fca-bc16-0a83a6f6a798"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:42:14.499132 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:14.499094    4871 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6832135c-e3d3-4fca-bc16-0a83a6f6a798-tls-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:42:14.508121 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:14.508083    4871 generic.go:356] "Generic (PLEG): container finished" podID="6832135c-e3d3-4fca-bc16-0a83a6f6a798" containerID="9c6ba627ba15fbaa1d88e90aeb81d6551e9b795f95bdacf573c349e862481b18" exitCode=0
Oct 30 22:42:14.508121 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:14.508134    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w1-ef00a4/auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s" event={"ID":"6832135c-e3d3-4fca-bc16-0a83a6f6a798","Type":"ContainerDied","Data":"9c6ba627ba15fbaa1d88e90aeb81d6551e9b795f95bdacf573c349e862481b18"}
Oct 30 22:42:14.508415 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:14.508140    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w1-ef00a4/auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s"
Oct 30 22:42:14.508415 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:14.508170    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w1-ef00a4/auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s" event={"ID":"6832135c-e3d3-4fca-bc16-0a83a6f6a798","Type":"ContainerDied","Data":"73f23078cd31428c3aa199c7c4285d87a28e0d7fe0e871ea346921587e00395f"}
Oct 30 22:42:14.508415 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:14.508196    4871 scope.go:122] "RemoveContainer" containerID="9c6ba627ba15fbaa1d88e90aeb81d6551e9b795f95bdacf573c349e862481b18"
Oct 30 22:42:14.520976 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:14.520949    4871 scope.go:122] "RemoveContainer" containerID="9c6ba627ba15fbaa1d88e90aeb81d6551e9b795f95bdacf573c349e862481b18"
Oct 30 22:42:14.521352 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:42:14.521331    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c6ba627ba15fbaa1d88e90aeb81d6551e9b795f95bdacf573c349e862481b18\": container with ID starting with 9c6ba627ba15fbaa1d88e90aeb81d6551e9b795f95bdacf573c349e862481b18 not found: ID does not exist" containerID="9c6ba627ba15fbaa1d88e90aeb81d6551e9b795f95bdacf573c349e862481b18"
Oct 30 22:42:14.521431 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:14.521365    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c6ba627ba15fbaa1d88e90aeb81d6551e9b795f95bdacf573c349e862481b18"} err="failed to get container status \"9c6ba627ba15fbaa1d88e90aeb81d6551e9b795f95bdacf573c349e862481b18\": rpc error: code = NotFound desc = could not find container \"9c6ba627ba15fbaa1d88e90aeb81d6551e9b795f95bdacf573c349e862481b18\": container with ID starting with 9c6ba627ba15fbaa1d88e90aeb81d6551e9b795f95bdacf573c349e862481b18 not found: ID does not exist"
Oct 30 22:42:14.683443 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:14.683405    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w1-ef00a4/auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s"]
Oct 30 22:42:14.690831 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:14.690784    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w1-ef00a4/auth-test-model-ef00a4-kserve-5b4c746dc7-tdl7s"]
Oct 30 22:42:15.702127 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:15.702087    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:15.962930 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:15.962842    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6832135c-e3d3-4fca-bc16-0a83a6f6a798" path="/var/lib/kubelet/pods/6832135c-e3d3-4fca-bc16-0a83a6f6a798/volumes"
Oct 30 22:42:20.302038 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:20.301983    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:24.828488 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:24.828445    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:29.461345 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:29.461311    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:34.041579 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:34.041539    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:34.201318 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:34.201287    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:34.365823 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:34.365793    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:35.568950 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:42:35.568909    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:42:38.777602 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:38.777569    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx_8d401654-8a4c-4a79-b2d3-612765312421/extract/0.log"
Oct 30 22:42:38.782336 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:38.782308    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx_8d401654-8a4c-4a79-b2d3-612765312421/extract/0.log"
Oct 30 22:42:38.790322 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:38.790270    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/2.log"
Oct 30 22:42:38.794679 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:38.794652    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/2.log"
Oct 30 22:42:38.797692 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:38.797667    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzntd_2360fddc-52a5-42a5-82ca-a05aff84d217/ovn-acl-logging/0.log"
Oct 30 22:42:38.802195 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:38.802175    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzntd_2360fddc-52a5-42a5-82ca-a05aff84d217/ovn-acl-logging/0.log"
Oct 30 22:42:38.825611 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:38.825568    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:40.904955 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:40.904924    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:41.530494 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:41.530405    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"]
Oct 30 22:42:41.531070 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:41.531055    4871 cpu_manager.go:414] "containerMap: removing container" podUID="e631c7a2-bd8e-4a9f-8768-6fa43cde6080" containerName="istio-proxy"
Oct 30 22:42:41.531114 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:41.531071    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e631c7a2-bd8e-4a9f-8768-6fa43cde6080" containerName="istio-proxy"
Oct 30 22:42:41.531114 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:41.531098    4871 cpu_manager.go:414] "containerMap: removing container" podUID="6832135c-e3d3-4fca-bc16-0a83a6f6a798" containerName="main"
Oct 30 22:42:41.531114 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:41.531103    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6832135c-e3d3-4fca-bc16-0a83a6f6a798" containerName="main"
Oct 30 22:42:41.531213 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:41.531205    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="e631c7a2-bd8e-4a9f-8768-6fa43cde6080" containerName="istio-proxy"
Oct 30 22:42:41.531292 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:41.531215    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="6832135c-e3d3-4fca-bc16-0a83a6f6a798" containerName="main"
Oct 30 22:42:41.536358 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:41.536333    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"
Oct 30 22:42:41.587356 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:41.587311    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-istio-token\") pod \"e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr\" (UID: \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\") " pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"
Oct 30 22:42:41.587650 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:41.587625    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-istio-envoy\") pod \"e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr\" (UID: \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\") " pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"
Oct 30 22:42:41.587830 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:41.587813    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-workload-certs\") pod \"e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr\" (UID: \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\") " pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"
Oct 30 22:42:41.588015 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:41.587998    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-istiod-ca-cert\") pod \"e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr\" (UID: \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\") " pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"
Oct 30 22:42:41.588167 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:41.588152    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-istio-podinfo\") pod \"e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr\" (UID: \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\") " pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"
Oct 30 22:42:41.588378 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:41.588360    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-workload-socket\") pod \"e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr\" (UID: \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\") " pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"
Oct 30 22:42:41.588572 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:41.588551    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-istio-data\") pod \"e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr\" (UID: \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\") " pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"
Oct 30 22:42:41.608434 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:41.608384    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqzhf\" (UniqueName: \"kubernetes.io/projected/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-kube-api-access-tqzhf\") pod \"e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr\" (UID: \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\") " pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"
Oct 30 22:42:41.608637 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:41.608449    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-credential-socket\") pod \"e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr\" (UID: \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\") " pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"
Oct 30 22:42:41.681264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:41.671793    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"]
Oct 30 22:42:41.696178 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:41.682883    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-be6ecc-openshift-default-dockercfg-xgq54\""
Oct 30 22:42:41.719304 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:41.712083    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-istio-data\") pod \"e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr\" (UID: \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\") " pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"
Oct 30 22:42:41.719750 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:41.719708    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqzhf\" (UniqueName: \"kubernetes.io/projected/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-kube-api-access-tqzhf\") pod \"e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr\" (UID: \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\") " pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"
Oct 30 22:42:41.720023 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:41.719989    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-credential-socket\") pod \"e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr\" (UID: \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\") " pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"
Oct 30 22:42:41.720293 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:41.720275    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-istio-token\") pod \"e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr\" (UID: \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\") " pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"
Oct 30 22:42:41.720477 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:41.720461    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-istio-envoy\") pod \"e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr\" (UID: \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\") " pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"
Oct 30 22:42:41.720709 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:41.720688    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-workload-certs\") pod \"e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr\" (UID: \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\") " pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"
Oct 30 22:42:41.720992 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:41.720974    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-istiod-ca-cert\") pod \"e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr\" (UID: \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\") " pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"
Oct 30 22:42:41.721199 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:41.721177    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-istio-podinfo\") pod \"e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr\" (UID: \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\") " pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"
Oct 30 22:42:41.721471 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:41.721452    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-workload-socket\") pod \"e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr\" (UID: \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\") " pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"
Oct 30 22:42:41.731874 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:41.722638    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-credential-socket\") pod \"e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr\" (UID: \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\") " pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"
Oct 30 22:42:41.731874 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:41.722927    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-istio-data\") pod \"e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr\" (UID: \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\") " pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"
Oct 30 22:42:41.731874 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:41.722987    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-workload-socket\") pod \"e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr\" (UID: \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\") " pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"
Oct 30 22:42:41.731874 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:41.725049    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-istiod-ca-cert\") pod \"e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr\" (UID: \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\") " pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"
Oct 30 22:42:41.731874 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:41.731595    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-istio-podinfo\") pod \"e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr\" (UID: \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\") " pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"
Oct 30 22:42:41.731874 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:41.731799    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-workload-certs\") pod \"e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr\" (UID: \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\") " pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"
Oct 30 22:42:41.784347 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:41.784274    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-istio-envoy\") pod \"e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr\" (UID: \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\") " pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"
Oct 30 22:42:41.789154 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:41.788564    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:41.822735 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:41.820314    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqzhf\" (UniqueName: \"kubernetes.io/projected/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-kube-api-access-tqzhf\") pod \"e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr\" (UID: \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\") " pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"
Oct 30 22:42:41.822735 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:41.822180    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-istio-token\") pod \"e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr\" (UID: \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\") " pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"
Oct 30 22:42:41.921140 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:41.921090    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"
Oct 30 22:42:41.988941 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:41.988903    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:42.363824 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:42.363778    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"]
Oct 30 22:42:42.386859 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:42:42.386827    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cb08cf4_430f_416a_b1f8_81b5695e3f4e.slice/crio-b6ad760178354114f16c353c135ada3f7f0ae6290beabd01cc741215cb5731c1 WatchSource:0}: Error finding container b6ad760178354114f16c353c135ada3f7f0ae6290beabd01cc741215cb5731c1: Status 404 returned error can't find the container with id b6ad760178354114f16c353c135ada3f7f0ae6290beabd01cc741215cb5731c1
Oct 30 22:42:42.389709 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:42.389676    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:42:42.389809 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:42.389751    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:42:42.389809 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:42.389784    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:42:42.831222 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:42.831100    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr" event={"ID":"1cb08cf4-430f-416a-b1f8-81b5695e3f4e","Type":"ContainerStarted","Data":"6d5103abf164ff54e5de46fdcd830550176dfbb336a4a6162e4e0f4327ea439c"}
Oct 30 22:42:42.831222 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:42.831171    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr" event={"ID":"1cb08cf4-430f-416a-b1f8-81b5695e3f4e","Type":"ContainerStarted","Data":"b6ad760178354114f16c353c135ada3f7f0ae6290beabd01cc741215cb5731c1"}
Oct 30 22:42:42.922382 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:42.922338    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"
Oct 30 22:42:42.963743 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:42.963687    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr" podStartSLOduration=1.963672359 podStartE2EDuration="1.963672359s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:42:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:42:42.959707957 +0000 UTC m=+3905.137196574" watchObservedRunningTime="2026-10-30 22:42:42.963672359 +0000 UTC m=+3905.141160976"
Oct 30 22:42:43.631903 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:43.631868    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:43.929185 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:43.929084    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"
Oct 30 22:42:44.847636 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:44.847600    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"
Oct 30 22:42:44.848693 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:44.848661    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"
Oct 30 22:42:48.242070 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:48.241980    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:51.173118 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:51.173054    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:51.372609 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:51.372567    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:51.764072 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:51.764033    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:52.591882 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:52.591814    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:52.908081 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:52.908041    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z"]
Oct 30 22:42:52.912888 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:52.912864    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z"
Oct 30 22:42:52.969523 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:52.969479    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z"]
Oct 30 22:42:53.015904 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:53.015856    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-a-1a5f4a-serving-cert\""
Oct 30 22:42:53.097217 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:53.097187    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:53.099546 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:53.099510    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b7e498c-c51e-4f65-8f1c-e1fb615f278c-maas-api-tls\") pod \"maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z\" (UID: \"9b7e498c-c51e-4f65-8f1c-e1fb615f278c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z"
Oct 30 22:42:53.099697 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:53.099616    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgv7q\" (UniqueName: \"kubernetes.io/projected/9b7e498c-c51e-4f65-8f1c-e1fb615f278c-kube-api-access-cgv7q\") pod \"maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z\" (UID: \"9b7e498c-c51e-4f65-8f1c-e1fb615f278c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z"
Oct 30 22:42:53.099962 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:53.099933    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b7e498c-c51e-4f65-8f1c-e1fb615f278c-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z\" (UID: \"9b7e498c-c51e-4f65-8f1c-e1fb615f278c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z"
Oct 30 22:42:53.100074 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:53.099995    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b7e498c-c51e-4f65-8f1c-e1fb615f278c-metrics-certs\") pod \"maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z\" (UID: \"9b7e498c-c51e-4f65-8f1c-e1fb615f278c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z"
Oct 30 22:42:53.201171 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:53.201083    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b7e498c-c51e-4f65-8f1c-e1fb615f278c-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z\" (UID: \"9b7e498c-c51e-4f65-8f1c-e1fb615f278c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z"
Oct 30 22:42:53.201171 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:53.201127    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b7e498c-c51e-4f65-8f1c-e1fb615f278c-metrics-certs\") pod \"maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z\" (UID: \"9b7e498c-c51e-4f65-8f1c-e1fb615f278c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z"
Oct 30 22:42:53.201423 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:53.201281    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b7e498c-c51e-4f65-8f1c-e1fb615f278c-maas-api-tls\") pod \"maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z\" (UID: \"9b7e498c-c51e-4f65-8f1c-e1fb615f278c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z"
Oct 30 22:42:53.201423 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:53.201361    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgv7q\" (UniqueName: \"kubernetes.io/projected/9b7e498c-c51e-4f65-8f1c-e1fb615f278c-kube-api-access-cgv7q\") pod \"maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z\" (UID: \"9b7e498c-c51e-4f65-8f1c-e1fb615f278c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z"
Oct 30 22:42:53.201423 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:42:53.201370    4871 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-a-1a5f4a-serving-cert: secret "maas-api-e2e-extmod-a-1a5f4a-serving-cert" not found
Oct 30 22:42:53.201550 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:42:53.201452    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b7e498c-c51e-4f65-8f1c-e1fb615f278c-maas-api-tls podName:9b7e498c-c51e-4f65-8f1c-e1fb615f278c nodeName:}" failed. No retries permitted until 2026-10-30 22:42:53.701430764 +0000 UTC m=+3915.878919375 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9b7e498c-c51e-4f65-8f1c-e1fb615f278c-maas-api-tls") pod "maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z" (UID: "9b7e498c-c51e-4f65-8f1c-e1fb615f278c") : secret "maas-api-e2e-extmod-a-1a5f4a-serving-cert" not found
Oct 30 22:42:53.201963 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:53.201943    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b7e498c-c51e-4f65-8f1c-e1fb615f278c-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z\" (UID: \"9b7e498c-c51e-4f65-8f1c-e1fb615f278c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z"
Oct 30 22:42:53.203779 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:53.203757    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b7e498c-c51e-4f65-8f1c-e1fb615f278c-metrics-certs\") pod \"maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z\" (UID: \"9b7e498c-c51e-4f65-8f1c-e1fb615f278c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z"
Oct 30 22:42:53.251175 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:53.251139    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:53.267419 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:53.267387    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgv7q\" (UniqueName: \"kubernetes.io/projected/9b7e498c-c51e-4f65-8f1c-e1fb615f278c-kube-api-access-cgv7q\") pod \"maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z\" (UID: \"9b7e498c-c51e-4f65-8f1c-e1fb615f278c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z"
Oct 30 22:42:53.405900 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:53.405775    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:53.710567 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:53.710528    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b7e498c-c51e-4f65-8f1c-e1fb615f278c-maas-api-tls\") pod \"maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z\" (UID: \"9b7e498c-c51e-4f65-8f1c-e1fb615f278c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z"
Oct 30 22:42:53.713098 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:53.713068    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b7e498c-c51e-4f65-8f1c-e1fb615f278c-maas-api-tls\") pod \"maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z\" (UID: \"9b7e498c-c51e-4f65-8f1c-e1fb615f278c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z"
Oct 30 22:42:53.829116 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:53.829070    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z"
Oct 30 22:42:54.209770 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:42:54.209731    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b7e498c_c51e_4f65_8f1c_e1fb615f278c.slice/crio-b66f046681fc056a9e0a10a63639724122b5f47d146c139470865fa13d8192aa WatchSource:0}: Error finding container b66f046681fc056a9e0a10a63639724122b5f47d146c139470865fa13d8192aa: Status 404 returned error can't find the container with id b66f046681fc056a9e0a10a63639724122b5f47d146c139470865fa13d8192aa
Oct 30 22:42:54.211520 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:54.211499    4871 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:42:54.216370 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:54.216343    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z"]
Oct 30 22:42:54.991283 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:54.985674    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z" event={"ID":"9b7e498c-c51e-4f65-8f1c-e1fb615f278c","Type":"ContainerStarted","Data":"b66f046681fc056a9e0a10a63639724122b5f47d146c139470865fa13d8192aa"}
Oct 30 22:42:55.652938 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:55.652899    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:55.879866 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:55.879834    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:56.030327 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:56.030034    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z" event={"ID":"9b7e498c-c51e-4f65-8f1c-e1fb615f278c","Type":"ContainerStarted","Data":"0cda477b231f4d2be9ef394e19dcce21503e1a4961852a3bfb86feeb329889e5"}
Oct 30 22:42:56.048259 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:56.031192    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z"
Oct 30 22:42:56.084295 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:56.075841    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:56.194685 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:56.194628    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z" podStartSLOduration=3.315683426 podStartE2EDuration="4.194611109s" totalImagesPullingTime="878.927683ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:42:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:42:56.190735996 +0000 UTC m=+3918.368224633" watchObservedRunningTime="2026-10-30 22:42:56.194611109 +0000 UTC m=+3918.372099728"
Oct 30 22:42:58.185415 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:42:58.185363    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:01.786684 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:43:01.786644    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:02.018511 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:43:02.018473    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:03.056195 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:43:03.056158    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z"
Oct 30 22:43:03.375171 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:43:03.375138    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:05.569772 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:43:05.569734    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:43:08.471020 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:43:08.470971    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:13.364587 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:43:13.364553    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:17.995650 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:43:17.995594    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:22.541510 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:43:22.541466    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:26.612927 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:43:26.612885    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:27.266695 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:43:27.266656    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:31.835716 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:43:31.835680    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:35.570074 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:43:35.570037    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:43:36.349502 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:43:36.349461    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:40.383058 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:43:40.383019    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:41.058261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:43:41.057809    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:41.310486 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:43:41.310408    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:42.717059 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:43:42.717021    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:43.612809 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:43:43.612766    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:44.084101 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:43:44.084015    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:44.257926 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:43:44.257897    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:44.790997 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:43:44.790965    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:44.962915 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:43:44.962881    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:45.445845 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:43:45.445810    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:45.665265 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:43:45.664888    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:46.191727 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:43:46.191694    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:49.169785 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:43:49.169749    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:49.331596 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:43:49.331558    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:49.496264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:43:49.496163    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:49.809178 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:43:49.809077    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:50.030928 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:43:50.030892    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:50.198982 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:43:50.198929    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:50.996262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:43:50.996215    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:51.153253 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:43:51.153208    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:51.520318 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:43:51.520281    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:52.392218 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:43:52.392184    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:52.579044 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:43:52.577878    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:52.734320 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:43:52.733393    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:53.057776 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:43:53.057656    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:55.920699 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:43:55.920660    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:57.479383 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:43:57.479348    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:57.653931 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:43:57.651266    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:00.531066 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:00.531023    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:05.203342 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:05.203300    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:05.574151 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:44:05.574064    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:44:07.144140 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:07.144104    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:09.502117 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:09.502075    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:10.204258 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:10.201616    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:10.477248 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:10.477145    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:10.672053 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:10.672017    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:14.307521 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:14.307483    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:14.504759 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:14.504721    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:14.658994 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:14.658950    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:15.184787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:15.184709    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:17.644775 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:17.644741    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:17.862295 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:17.862260    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:20.277073 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:20.277040    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:22.716620 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:22.716570    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:23.599936 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:23.599895    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:25.201865 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:25.201831    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:25.363586 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:25.363548    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:26.456050 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:26.456018    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:27.853368 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:27.853331    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:28.375621 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:28.375587    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"]
Oct 30 22:44:28.380072 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:28.380053    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"
Oct 30 22:44:28.426989 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:28.426942    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38bc129a-15c0-4019-b32d-e4b62e79b513-workload-socket\") pod \"e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm\" (UID: \"38bc129a-15c0-4019-b32d-e4b62e79b513\") " pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"
Oct 30 22:44:28.427296 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:28.427269    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38bc129a-15c0-4019-b32d-e4b62e79b513-istio-envoy\") pod \"e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm\" (UID: \"38bc129a-15c0-4019-b32d-e4b62e79b513\") " pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"
Oct 30 22:44:28.427495 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:28.427475    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38bc129a-15c0-4019-b32d-e4b62e79b513-istio-data\") pod \"e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm\" (UID: \"38bc129a-15c0-4019-b32d-e4b62e79b513\") " pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"
Oct 30 22:44:28.427709 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:28.427691    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38bc129a-15c0-4019-b32d-e4b62e79b513-workload-certs\") pod \"e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm\" (UID: \"38bc129a-15c0-4019-b32d-e4b62e79b513\") " pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"
Oct 30 22:44:28.427892 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:28.427876    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38bc129a-15c0-4019-b32d-e4b62e79b513-credential-socket\") pod \"e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm\" (UID: \"38bc129a-15c0-4019-b32d-e4b62e79b513\") " pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"
Oct 30 22:44:28.428030 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:28.428015    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38bc129a-15c0-4019-b32d-e4b62e79b513-istio-token\") pod \"e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm\" (UID: \"38bc129a-15c0-4019-b32d-e4b62e79b513\") " pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"
Oct 30 22:44:28.428176 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:28.428162    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38bc129a-15c0-4019-b32d-e4b62e79b513-istio-podinfo\") pod \"e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm\" (UID: \"38bc129a-15c0-4019-b32d-e4b62e79b513\") " pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"
Oct 30 22:44:28.428497 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:28.428476    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzlf6\" (UniqueName: \"kubernetes.io/projected/38bc129a-15c0-4019-b32d-e4b62e79b513-kube-api-access-qzlf6\") pod \"e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm\" (UID: \"38bc129a-15c0-4019-b32d-e4b62e79b513\") " pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"
Oct 30 22:44:28.433576 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:28.433546    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38bc129a-15c0-4019-b32d-e4b62e79b513-istiod-ca-cert\") pod \"e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm\" (UID: \"38bc129a-15c0-4019-b32d-e4b62e79b513\") " pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"
Oct 30 22:44:28.631366 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:28.486314    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"]
Oct 30 22:44:28.631867 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:28.508818    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-b-6b46c3-openshift-default-dockercfg-ctk79\""
Oct 30 22:44:28.632038 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:28.519911    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:28.632224 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:28.580635    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38bc129a-15c0-4019-b32d-e4b62e79b513-istio-data\") pod \"e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm\" (UID: \"38bc129a-15c0-4019-b32d-e4b62e79b513\") " pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"
Oct 30 22:44:28.632541 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:28.632519    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38bc129a-15c0-4019-b32d-e4b62e79b513-workload-certs\") pod \"e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm\" (UID: \"38bc129a-15c0-4019-b32d-e4b62e79b513\") " pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"
Oct 30 22:44:28.632773 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:28.632749    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38bc129a-15c0-4019-b32d-e4b62e79b513-credential-socket\") pod \"e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm\" (UID: \"38bc129a-15c0-4019-b32d-e4b62e79b513\") " pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"
Oct 30 22:44:28.632949 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:28.632934    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38bc129a-15c0-4019-b32d-e4b62e79b513-istio-token\") pod \"e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm\" (UID: \"38bc129a-15c0-4019-b32d-e4b62e79b513\") " pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"
Oct 30 22:44:28.633124 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:28.633092    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38bc129a-15c0-4019-b32d-e4b62e79b513-istio-podinfo\") pod \"e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm\" (UID: \"38bc129a-15c0-4019-b32d-e4b62e79b513\") " pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"
Oct 30 22:44:28.633263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:28.633231    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzlf6\" (UniqueName: \"kubernetes.io/projected/38bc129a-15c0-4019-b32d-e4b62e79b513-kube-api-access-qzlf6\") pod \"e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm\" (UID: \"38bc129a-15c0-4019-b32d-e4b62e79b513\") " pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"
Oct 30 22:44:28.633409 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:28.633380    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38bc129a-15c0-4019-b32d-e4b62e79b513-istiod-ca-cert\") pod \"e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm\" (UID: \"38bc129a-15c0-4019-b32d-e4b62e79b513\") " pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"
Oct 30 22:44:28.633519 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:28.633507    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38bc129a-15c0-4019-b32d-e4b62e79b513-workload-socket\") pod \"e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm\" (UID: \"38bc129a-15c0-4019-b32d-e4b62e79b513\") " pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"
Oct 30 22:44:28.633596 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:28.633587    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38bc129a-15c0-4019-b32d-e4b62e79b513-istio-envoy\") pod \"e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm\" (UID: \"38bc129a-15c0-4019-b32d-e4b62e79b513\") " pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"
Oct 30 22:44:28.636708 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:28.583318    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38bc129a-15c0-4019-b32d-e4b62e79b513-istio-data\") pod \"e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm\" (UID: \"38bc129a-15c0-4019-b32d-e4b62e79b513\") " pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"
Oct 30 22:44:28.636708 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:28.634972    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38bc129a-15c0-4019-b32d-e4b62e79b513-workload-certs\") pod \"e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm\" (UID: \"38bc129a-15c0-4019-b32d-e4b62e79b513\") " pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"
Oct 30 22:44:28.636708 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:28.635308    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38bc129a-15c0-4019-b32d-e4b62e79b513-credential-socket\") pod \"e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm\" (UID: \"38bc129a-15c0-4019-b32d-e4b62e79b513\") " pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"
Oct 30 22:44:28.639616 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:28.639594    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38bc129a-15c0-4019-b32d-e4b62e79b513-workload-socket\") pod \"e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm\" (UID: \"38bc129a-15c0-4019-b32d-e4b62e79b513\") " pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"
Oct 30 22:44:28.640332 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:28.640255    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38bc129a-15c0-4019-b32d-e4b62e79b513-istiod-ca-cert\") pod \"e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm\" (UID: \"38bc129a-15c0-4019-b32d-e4b62e79b513\") " pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"
Oct 30 22:44:28.642596 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:28.642572    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38bc129a-15c0-4019-b32d-e4b62e79b513-istio-podinfo\") pod \"e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm\" (UID: \"38bc129a-15c0-4019-b32d-e4b62e79b513\") " pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"
Oct 30 22:44:28.653201 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:28.651521    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38bc129a-15c0-4019-b32d-e4b62e79b513-istio-envoy\") pod \"e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm\" (UID: \"38bc129a-15c0-4019-b32d-e4b62e79b513\") " pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"
Oct 30 22:44:28.705440 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:28.705406    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38bc129a-15c0-4019-b32d-e4b62e79b513-istio-token\") pod \"e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm\" (UID: \"38bc129a-15c0-4019-b32d-e4b62e79b513\") " pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"
Oct 30 22:44:28.708561 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:28.708529    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzlf6\" (UniqueName: \"kubernetes.io/projected/38bc129a-15c0-4019-b32d-e4b62e79b513-kube-api-access-qzlf6\") pod \"e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm\" (UID: \"38bc129a-15c0-4019-b32d-e4b62e79b513\") " pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"
Oct 30 22:44:28.858455 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:28.858420    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:28.995384 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:28.995302    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"
Oct 30 22:44:29.459139 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:44:29.459103    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38bc129a_15c0_4019_b32d_e4b62e79b513.slice/crio-124792a0cd3f99548824faa7cc6d5e6e6081a7dbbe7dd52744332f609bda5b2e WatchSource:0}: Error finding container 124792a0cd3f99548824faa7cc6d5e6e6081a7dbbe7dd52744332f609bda5b2e: Status 404 returned error can't find the container with id 124792a0cd3f99548824faa7cc6d5e6e6081a7dbbe7dd52744332f609bda5b2e
Oct 30 22:44:29.462076 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:29.462040    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:44:29.462189 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:29.462127    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:44:29.462189 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:29.462173    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:44:29.465163 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:29.465141    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"]
Oct 30 22:44:29.975963 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:29.975929    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm" event={"ID":"38bc129a-15c0-4019-b32d-e4b62e79b513","Type":"ContainerStarted","Data":"b94fbf535e2488f0a0a0edf651ee20c0a96d29d6a32132410586abaeaddf90d1"}
Oct 30 22:44:29.976444 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:29.975992    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm" event={"ID":"38bc129a-15c0-4019-b32d-e4b62e79b513","Type":"ContainerStarted","Data":"124792a0cd3f99548824faa7cc6d5e6e6081a7dbbe7dd52744332f609bda5b2e"}
Oct 30 22:44:29.996375 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:29.996340    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"
Oct 30 22:44:30.107998 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:30.107938    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm" podStartSLOduration=2.107922959 podStartE2EDuration="2.107922959s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:44:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:44:30.104005109 +0000 UTC m=+4012.281493720" watchObservedRunningTime="2026-10-30 22:44:30.107922959 +0000 UTC m=+4012.285411575"
Oct 30 22:44:30.438310 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:30.438266    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:31.001897 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:31.001864    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"
Oct 30 22:44:31.991719 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:31.991686    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"
Oct 30 22:44:31.992740 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:31.992722    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"
Oct 30 22:44:32.763933 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:32.763889    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:32.963505 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:32.963456    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:33.141978 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:33.141941    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:35.478693 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:35.478654    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:35.573293 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:44:35.573231    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:44:35.973745 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:35.973708    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:36.168521 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:36.162981    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:40.315179 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:40.315125    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:45.120069 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:45.120013    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:49.729287 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:49.729225    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:51.895811 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:51.895761    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:53.429142 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:53.429084    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:54.368136 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:54.368079    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:58.216617 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:58.216529    4871 generic.go:356] "Generic (PLEG): container finished" podID="1cb08cf4-430f-416a-b1f8-81b5695e3f4e" containerID="6d5103abf164ff54e5de46fdcd830550176dfbb336a4a6162e4e0f4327ea439c" exitCode=0
Oct 30 22:44:58.216617 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:58.216585    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr" event={"ID":"1cb08cf4-430f-416a-b1f8-81b5695e3f4e","Type":"ContainerDied","Data":"6d5103abf164ff54e5de46fdcd830550176dfbb336a4a6162e4e0f4327ea439c"}
Oct 30 22:44:58.217156 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:58.217012    4871 scope.go:122] "RemoveContainer" containerID="6d5103abf164ff54e5de46fdcd830550176dfbb336a4a6162e4e0f4327ea439c"
Oct 30 22:44:58.218741 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:58.218710    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:44:58.218826 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:58.218784    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:44:58.218826 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:58.218815    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:44:58.929887 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:58.929847    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:59.241972 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:59.241855    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr" event={"ID":"1cb08cf4-430f-416a-b1f8-81b5695e3f4e","Type":"ContainerStarted","Data":"12dc24aea3a707a18fc5229f060f96992fef4705a4d148eef6e3d6fcb7c5b08b"}
Oct 30 22:44:59.848516 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:59.848468    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"
Oct 30 22:44:59.926264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:59.925562    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"
Oct 30 22:44:59.928468 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:59.928439    4871 patch_prober.go:29] interesting pod/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.195:15021/healthz/ready\": dial tcp 10.133.0.195:15021: connect: connection refused" start-of-body=
Oct 30 22:44:59.928641 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:44:59.928492    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr" podUID="1cb08cf4-430f-416a-b1f8-81b5695e3f4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.195:15021/healthz/ready\": dial tcp 10.133.0.195:15021: connect: connection refused"
Oct 30 22:45:00.277724 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:00.277687    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-43d592-29890005-sxcjg"]
Oct 30 22:45:00.281699 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:00.281672    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-43d592-29890005-sxcjg"
Oct 30 22:45:00.322508 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:00.319297    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-8c852b-29890005-2vxbn"]
Oct 30 22:45:00.323990 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:00.323958    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-be6ecc-29890005-wsbxj"]
Oct 30 22:45:00.340507 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:00.340467    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-8c852b-29890005-2vxbn"
Oct 30 22:45:00.343095 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:00.342974    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-be6ecc-29890005-wsbxj"
Oct 30 22:45:00.368893 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:00.368856    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-43d592-29890005-sxcjg"]
Oct 30 22:45:00.375323 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:00.375295    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mlvtf\""
Oct 30 22:45:00.426261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:00.402669    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxtgz\" (UniqueName: \"kubernetes.io/projected/752747f8-66a0-4631-98e8-e9207c22916f-kube-api-access-wxtgz\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-43d592-29890005-sxcjg\" (UID: \"752747f8-66a0-4631-98e8-e9207c22916f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-43d592-29890005-sxcjg"
Oct 30 22:45:00.434264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:00.426753    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/752747f8-66a0-4631-98e8-e9207c22916f-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-43d592-29890005-sxcjg\" (UID: \"752747f8-66a0-4631-98e8-e9207c22916f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-43d592-29890005-sxcjg"
Oct 30 22:45:00.456791 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:00.456745    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-8c852b-29890005-2vxbn"]
Oct 30 22:45:00.461525 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:00.461450    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-be6ecc-29890005-wsbxj"]
Oct 30 22:45:00.552732 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:00.552634    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/752747f8-66a0-4631-98e8-e9207c22916f-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-43d592-29890005-sxcjg\" (UID: \"752747f8-66a0-4631-98e8-e9207c22916f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-43d592-29890005-sxcjg"
Oct 30 22:45:00.552732 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:00.552694    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppdzd\" (UniqueName: \"kubernetes.io/projected/fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2-kube-api-access-ppdzd\") pod \"maas-api-key-cleanup-e2e-extmod-a-8c852b-29890005-2vxbn\" (UID: \"fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-8c852b-29890005-2vxbn"
Oct 30 22:45:00.552999 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:00.552877    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jp9m8\" (UniqueName: \"kubernetes.io/projected/61d9910d-4558-4ade-a332-126ca461a006-kube-api-access-jp9m8\") pod \"maas-api-key-cleanup-e2e-extmod-a-be6ecc-29890005-wsbxj\" (UID: \"61d9910d-4558-4ade-a332-126ca461a006\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-be6ecc-29890005-wsbxj"
Oct 30 22:45:00.552999 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:00.552946    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxtgz\" (UniqueName: \"kubernetes.io/projected/752747f8-66a0-4631-98e8-e9207c22916f-kube-api-access-wxtgz\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-43d592-29890005-sxcjg\" (UID: \"752747f8-66a0-4631-98e8-e9207c22916f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-43d592-29890005-sxcjg"
Oct 30 22:45:00.553114 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:00.553015    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/61d9910d-4558-4ade-a332-126ca461a006-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-be6ecc-29890005-wsbxj\" (UID: \"61d9910d-4558-4ade-a332-126ca461a006\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-be6ecc-29890005-wsbxj"
Oct 30 22:45:00.553182 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:00.553158    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-8c852b-29890005-2vxbn\" (UID: \"fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-8c852b-29890005-2vxbn"
Oct 30 22:45:00.553431 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:00.553403    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/752747f8-66a0-4631-98e8-e9207c22916f-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-43d592-29890005-sxcjg\" (UID: \"752747f8-66a0-4631-98e8-e9207c22916f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-43d592-29890005-sxcjg"
Oct 30 22:45:00.634139 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:00.634099    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxtgz\" (UniqueName: \"kubernetes.io/projected/752747f8-66a0-4631-98e8-e9207c22916f-kube-api-access-wxtgz\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-43d592-29890005-sxcjg\" (UID: \"752747f8-66a0-4631-98e8-e9207c22916f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-43d592-29890005-sxcjg"
Oct 30 22:45:00.655068 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:00.655022    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jp9m8\" (UniqueName: \"kubernetes.io/projected/61d9910d-4558-4ade-a332-126ca461a006-kube-api-access-jp9m8\") pod \"maas-api-key-cleanup-e2e-extmod-a-be6ecc-29890005-wsbxj\" (UID: \"61d9910d-4558-4ade-a332-126ca461a006\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-be6ecc-29890005-wsbxj"
Oct 30 22:45:00.655304 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:00.655118    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/61d9910d-4558-4ade-a332-126ca461a006-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-be6ecc-29890005-wsbxj\" (UID: \"61d9910d-4558-4ade-a332-126ca461a006\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-be6ecc-29890005-wsbxj"
Oct 30 22:45:00.655304 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:00.655199    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-8c852b-29890005-2vxbn\" (UID: \"fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-8c852b-29890005-2vxbn"
Oct 30 22:45:00.655456 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:00.655306    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppdzd\" (UniqueName: \"kubernetes.io/projected/fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2-kube-api-access-ppdzd\") pod \"maas-api-key-cleanup-e2e-extmod-a-8c852b-29890005-2vxbn\" (UID: \"fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-8c852b-29890005-2vxbn"
Oct 30 22:45:00.655861 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:00.655835    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/61d9910d-4558-4ade-a332-126ca461a006-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-be6ecc-29890005-wsbxj\" (UID: \"61d9910d-4558-4ade-a332-126ca461a006\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-be6ecc-29890005-wsbxj"
Oct 30 22:45:00.655942 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:00.655918    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-8c852b-29890005-2vxbn\" (UID: \"fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-8c852b-29890005-2vxbn"
Oct 30 22:45:00.734229 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:00.734192    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jp9m8\" (UniqueName: \"kubernetes.io/projected/61d9910d-4558-4ade-a332-126ca461a006-kube-api-access-jp9m8\") pod \"maas-api-key-cleanup-e2e-extmod-a-be6ecc-29890005-wsbxj\" (UID: \"61d9910d-4558-4ade-a332-126ca461a006\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-be6ecc-29890005-wsbxj"
Oct 30 22:45:00.736211 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:00.736186    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppdzd\" (UniqueName: \"kubernetes.io/projected/fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2-kube-api-access-ppdzd\") pod \"maas-api-key-cleanup-e2e-extmod-a-8c852b-29890005-2vxbn\" (UID: \"fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-8c852b-29890005-2vxbn"
Oct 30 22:45:00.896169 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:00.896118    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-43d592-29890005-sxcjg"
Oct 30 22:45:00.931910 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:00.931850    4871 patch_prober.go:29] interesting pod/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.195:15021/healthz/ready\": dial tcp 10.133.0.195:15021: connect: connection refused" start-of-body=
Oct 30 22:45:00.932337 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:00.931933    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr" podUID="1cb08cf4-430f-416a-b1f8-81b5695e3f4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.195:15021/healthz/ready\": dial tcp 10.133.0.195:15021: connect: connection refused"
Oct 30 22:45:00.973682 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:00.973637    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-8c852b-29890005-2vxbn"
Oct 30 22:45:00.985955 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:00.985424    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-be6ecc-29890005-wsbxj"
Oct 30 22:45:01.341393 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:45:01.341361    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod752747f8_66a0_4631_98e8_e9207c22916f.slice/crio-9139533b8ce1f51f5442941c3262e83384d7b28d23076713834f44e8a966bafd WatchSource:0}: Error finding container 9139533b8ce1f51f5442941c3262e83384d7b28d23076713834f44e8a966bafd: Status 404 returned error can't find the container with id 9139533b8ce1f51f5442941c3262e83384d7b28d23076713834f44e8a966bafd
Oct 30 22:45:01.346528 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:01.346497    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-43d592-29890005-sxcjg"]
Oct 30 22:45:01.468285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:01.461390    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-8c852b-29890005-2vxbn"]
Oct 30 22:45:01.536264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:01.535710    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-be6ecc-29890005-wsbxj"]
Oct 30 22:45:01.935026 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:01.934382    4871 patch_prober.go:29] interesting pod/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.195:15021/healthz/ready\": dial tcp 10.133.0.195:15021: connect: connection refused" start-of-body=
Oct 30 22:45:01.935026 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:01.934448    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr" podUID="1cb08cf4-430f-416a-b1f8-81b5695e3f4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.195:15021/healthz/ready\": dial tcp 10.133.0.195:15021: connect: connection refused"
Oct 30 22:45:02.276799 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:02.276752    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-43d592-29890005-sxcjg" event={"ID":"752747f8-66a0-4631-98e8-e9207c22916f","Type":"ContainerStarted","Data":"9cc9c406d59272f828ed5e465d8781e843b88174510a2ff0d6a2b823daf7a281"}
Oct 30 22:45:02.277004 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:02.276819    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-43d592-29890005-sxcjg" event={"ID":"752747f8-66a0-4631-98e8-e9207c22916f","Type":"ContainerStarted","Data":"9139533b8ce1f51f5442941c3262e83384d7b28d23076713834f44e8a966bafd"}
Oct 30 22:45:02.279771 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:02.278779    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-8c852b-29890005-2vxbn" event={"ID":"fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2","Type":"ContainerStarted","Data":"e38d00ba3c3a33b7bb9259827f463e8b420c9907a5bce0d7ca2e87a79f730852"}
Oct 30 22:45:02.279771 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:02.278845    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-8c852b-29890005-2vxbn" event={"ID":"fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2","Type":"ContainerStarted","Data":"7a703ab101114c9d1fb73ec1c0907fbbb4beb1c51789d2fdd1b3caf0b929b17f"}
Oct 30 22:45:02.280554 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:02.280533    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-be6ecc-29890005-wsbxj" event={"ID":"61d9910d-4558-4ade-a332-126ca461a006","Type":"ContainerStarted","Data":"ed6d963998aaa53fcda0182211b368ae354b337ed48d7fe4f30dc4c08f7618d9"}
Oct 30 22:45:02.280632 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:02.280572    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-be6ecc-29890005-wsbxj" event={"ID":"61d9910d-4558-4ade-a332-126ca461a006","Type":"ContainerStarted","Data":"71c2f992ac8448fea299207c16f5203c8ed48298a6c4c74f5ea16c249781ca40"}
Oct 30 22:45:02.416296 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:02.412911    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-43d592-29890005-sxcjg" podStartSLOduration=2.412890253 podStartE2EDuration="2.412890253s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:45:02.401000604 +0000 UTC m=+4044.578489221" watchObservedRunningTime="2026-10-30 22:45:02.412890253 +0000 UTC m=+4044.590378873"
Oct 30 22:45:02.549179 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:02.549114    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-8c852b-29890005-2vxbn" podStartSLOduration=2.549095716 podStartE2EDuration="2.549095716s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:45:02.53958976 +0000 UTC m=+4044.717078402" watchObservedRunningTime="2026-10-30 22:45:02.549095716 +0000 UTC m=+4044.726584333"
Oct 30 22:45:02.662800 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:02.662742    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-be6ecc-29890005-wsbxj" podStartSLOduration=2.662725832 podStartE2EDuration="2.662725832s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:45:02.661438654 +0000 UTC m=+4044.838927273" watchObservedRunningTime="2026-10-30 22:45:02.662725832 +0000 UTC m=+4044.840214450"
Oct 30 22:45:02.926262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:02.923774    4871 patch_prober.go:29] interesting pod/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.195:15021/healthz/ready\": dial tcp 10.133.0.195:15021: connect: connection refused" start-of-body=
Oct 30 22:45:02.926262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:02.923845    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr" podUID="1cb08cf4-430f-416a-b1f8-81b5695e3f4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.195:15021/healthz/ready\": dial tcp 10.133.0.195:15021: connect: connection refused"
Oct 30 22:45:03.290773 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:03.290740    4871 generic.go:356] "Generic (PLEG): container finished" podID="61d9910d-4558-4ade-a332-126ca461a006" containerID="ed6d963998aaa53fcda0182211b368ae354b337ed48d7fe4f30dc4c08f7618d9" exitCode=60
Oct 30 22:45:03.291182 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:03.291151    4871 scope.go:122] "RemoveContainer" containerID="ed6d963998aaa53fcda0182211b368ae354b337ed48d7fe4f30dc4c08f7618d9"
Oct 30 22:45:03.291313 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:03.290825    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-be6ecc-29890005-wsbxj" event={"ID":"61d9910d-4558-4ade-a332-126ca461a006","Type":"ContainerDied","Data":"ed6d963998aaa53fcda0182211b368ae354b337ed48d7fe4f30dc4c08f7618d9"}
Oct 30 22:45:03.294521 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:03.294495    4871 generic.go:356] "Generic (PLEG): container finished" podID="752747f8-66a0-4631-98e8-e9207c22916f" containerID="9cc9c406d59272f828ed5e465d8781e843b88174510a2ff0d6a2b823daf7a281" exitCode=60
Oct 30 22:45:03.294620 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:03.294572    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-43d592-29890005-sxcjg" event={"ID":"752747f8-66a0-4631-98e8-e9207c22916f","Type":"ContainerDied","Data":"9cc9c406d59272f828ed5e465d8781e843b88174510a2ff0d6a2b823daf7a281"}
Oct 30 22:45:03.295341 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:03.295324    4871 scope.go:122] "RemoveContainer" containerID="9cc9c406d59272f828ed5e465d8781e843b88174510a2ff0d6a2b823daf7a281"
Oct 30 22:45:03.296432 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:03.296408    4871 generic.go:356] "Generic (PLEG): container finished" podID="fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2" containerID="e38d00ba3c3a33b7bb9259827f463e8b420c9907a5bce0d7ca2e87a79f730852" exitCode=60
Oct 30 22:45:03.296530 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:03.296510    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-8c852b-29890005-2vxbn" event={"ID":"fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2","Type":"ContainerDied","Data":"e38d00ba3c3a33b7bb9259827f463e8b420c9907a5bce0d7ca2e87a79f730852"}
Oct 30 22:45:03.296774 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:03.296743    4871 scope.go:122] "RemoveContainer" containerID="e38d00ba3c3a33b7bb9259827f463e8b420c9907a5bce0d7ca2e87a79f730852"
Oct 30 22:45:03.727285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:03.726526    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:03.928913 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:03.928862    4871 patch_prober.go:29] interesting pod/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.195:15021/healthz/ready\": dial tcp 10.133.0.195:15021: connect: connection refused" start-of-body=
Oct 30 22:45:03.929211 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:03.929137    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr" podUID="1cb08cf4-430f-416a-b1f8-81b5695e3f4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.195:15021/healthz/ready\": dial tcp 10.133.0.195:15021: connect: connection refused"
Oct 30 22:45:04.307889 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:04.307844    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-43d592-29890005-sxcjg" event={"ID":"752747f8-66a0-4631-98e8-e9207c22916f","Type":"ContainerStarted","Data":"1eca7acd9bd68a64b539189be29d4b3caa2ab1f0695182d1894617a8afaf538f"}
Oct 30 22:45:04.310639 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:04.310605    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-8c852b-29890005-2vxbn" event={"ID":"fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2","Type":"ContainerStarted","Data":"63551e1791a3142896c506f34925027a69842ed2210dd3a7015e82625ba2b621"}
Oct 30 22:45:04.312943 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:04.312911    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-be6ecc-29890005-wsbxj" event={"ID":"61d9910d-4558-4ade-a332-126ca461a006","Type":"ContainerStarted","Data":"6eb23fccbe7154e7a12d3a6e561acae735a4996eba61948b7db4d2b273da861c"}
Oct 30 22:45:04.929371 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:04.929335    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"
Oct 30 22:45:04.930754 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:04.930729    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"
Oct 30 22:45:05.324186 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:05.324100    4871 generic.go:356] "Generic (PLEG): container finished" podID="752747f8-66a0-4631-98e8-e9207c22916f" containerID="1eca7acd9bd68a64b539189be29d4b3caa2ab1f0695182d1894617a8afaf538f" exitCode=60
Oct 30 22:45:05.324393 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:05.324182    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-43d592-29890005-sxcjg" event={"ID":"752747f8-66a0-4631-98e8-e9207c22916f","Type":"ContainerDied","Data":"1eca7acd9bd68a64b539189be29d4b3caa2ab1f0695182d1894617a8afaf538f"}
Oct 30 22:45:05.324393 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:05.324276    4871 scope.go:122] "RemoveContainer" containerID="9cc9c406d59272f828ed5e465d8781e843b88174510a2ff0d6a2b823daf7a281"
Oct 30 22:45:05.324722 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:05.324698    4871 scope.go:122] "RemoveContainer" containerID="1eca7acd9bd68a64b539189be29d4b3caa2ab1f0695182d1894617a8afaf538f"
Oct 30 22:45:05.325151 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:45:05.325120    4871 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w3-43d592-29890005-sxcjg_odh-ai-gateway-infra(752747f8-66a0-4631-98e8-e9207c22916f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-43d592-29890005-sxcjg" podUID="752747f8-66a0-4631-98e8-e9207c22916f"
Oct 30 22:45:05.327356 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:05.327330    4871 generic.go:356] "Generic (PLEG): container finished" podID="fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2" containerID="63551e1791a3142896c506f34925027a69842ed2210dd3a7015e82625ba2b621" exitCode=60
Oct 30 22:45:05.327449 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:05.327374    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-8c852b-29890005-2vxbn" event={"ID":"fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2","Type":"ContainerDied","Data":"63551e1791a3142896c506f34925027a69842ed2210dd3a7015e82625ba2b621"}
Oct 30 22:45:05.327796 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:05.327777    4871 scope.go:122] "RemoveContainer" containerID="63551e1791a3142896c506f34925027a69842ed2210dd3a7015e82625ba2b621"
Oct 30 22:45:05.328135 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:45:05.328108    4871 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-8c852b-29890005-2vxbn_odh-ai-gateway-infra(fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-8c852b-29890005-2vxbn" podUID="fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2"
Oct 30 22:45:05.330010 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:05.329984    4871 generic.go:356] "Generic (PLEG): container finished" podID="61d9910d-4558-4ade-a332-126ca461a006" containerID="6eb23fccbe7154e7a12d3a6e561acae735a4996eba61948b7db4d2b273da861c" exitCode=60
Oct 30 22:45:05.330118 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:05.330065    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-be6ecc-29890005-wsbxj" event={"ID":"61d9910d-4558-4ade-a332-126ca461a006","Type":"ContainerDied","Data":"6eb23fccbe7154e7a12d3a6e561acae735a4996eba61948b7db4d2b273da861c"}
Oct 30 22:45:05.330459 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:05.330444    4871 scope.go:122] "RemoveContainer" containerID="6eb23fccbe7154e7a12d3a6e561acae735a4996eba61948b7db4d2b273da861c"
Oct 30 22:45:05.330758 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:45:05.330739    4871 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-be6ecc-29890005-wsbxj_odh-ai-gateway-infra(61d9910d-4558-4ade-a332-126ca461a006)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-be6ecc-29890005-wsbxj" podUID="61d9910d-4558-4ade-a332-126ca461a006"
Oct 30 22:45:05.339414 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:05.339387    4871 scope.go:122] "RemoveContainer" containerID="e38d00ba3c3a33b7bb9259827f463e8b420c9907a5bce0d7ca2e87a79f730852"
Oct 30 22:45:05.358017 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:05.357985    4871 scope.go:122] "RemoveContainer" containerID="ed6d963998aaa53fcda0182211b368ae354b337ed48d7fe4f30dc4c08f7618d9"
Oct 30 22:45:05.571902 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:45:05.571863    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:45:06.340955 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:06.340923    4871 scope.go:122] "RemoveContainer" containerID="1eca7acd9bd68a64b539189be29d4b3caa2ab1f0695182d1894617a8afaf538f"
Oct 30 22:45:06.341500 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:45:06.341345    4871 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w3-43d592-29890005-sxcjg_odh-ai-gateway-infra(752747f8-66a0-4631-98e8-e9207c22916f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-43d592-29890005-sxcjg" podUID="752747f8-66a0-4631-98e8-e9207c22916f"
Oct 30 22:45:06.343075 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:06.343045    4871 scope.go:122] "RemoveContainer" containerID="63551e1791a3142896c506f34925027a69842ed2210dd3a7015e82625ba2b621"
Oct 30 22:45:06.343462 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:45:06.343435    4871 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-8c852b-29890005-2vxbn_odh-ai-gateway-infra(fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-8c852b-29890005-2vxbn" podUID="fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2"
Oct 30 22:45:06.345621 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:06.345598    4871 scope.go:122] "RemoveContainer" containerID="6eb23fccbe7154e7a12d3a6e561acae735a4996eba61948b7db4d2b273da861c"
Oct 30 22:45:06.345957 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:45:06.345935    4871 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-be6ecc-29890005-wsbxj_odh-ai-gateway-infra(61d9910d-4558-4ade-a332-126ca461a006)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-be6ecc-29890005-wsbxj" podUID="61d9910d-4558-4ade-a332-126ca461a006"
Oct 30 22:45:08.403586 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:08.403526    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:12.916476 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:12.916435    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:15.954131 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:15.954096    4871 scope.go:122] "RemoveContainer" containerID="1eca7acd9bd68a64b539189be29d4b3caa2ab1f0695182d1894617a8afaf538f"
Oct 30 22:45:16.428966 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:16.428930    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-43d592-29890005-sxcjg" event={"ID":"752747f8-66a0-4631-98e8-e9207c22916f","Type":"ContainerStarted","Data":"7ece90167ba6e39faad07fb451ce12c3b4d70d781f335b8302f2f0d3a02341bb"}
Oct 30 22:45:16.953030 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:16.952988    4871 scope.go:122] "RemoveContainer" containerID="63551e1791a3142896c506f34925027a69842ed2210dd3a7015e82625ba2b621"
Oct 30 22:45:16.953318 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:16.953298    4871 scope.go:122] "RemoveContainer" containerID="6eb23fccbe7154e7a12d3a6e561acae735a4996eba61948b7db4d2b273da861c"
Oct 30 22:45:17.441285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:17.441228    4871 generic.go:356] "Generic (PLEG): container finished" podID="752747f8-66a0-4631-98e8-e9207c22916f" containerID="7ece90167ba6e39faad07fb451ce12c3b4d70d781f335b8302f2f0d3a02341bb" exitCode=60
Oct 30 22:45:17.441805 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:17.441301    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-43d592-29890005-sxcjg" event={"ID":"752747f8-66a0-4631-98e8-e9207c22916f","Type":"ContainerDied","Data":"7ece90167ba6e39faad07fb451ce12c3b4d70d781f335b8302f2f0d3a02341bb"}
Oct 30 22:45:17.441805 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:17.441366    4871 scope.go:122] "RemoveContainer" containerID="1eca7acd9bd68a64b539189be29d4b3caa2ab1f0695182d1894617a8afaf538f"
Oct 30 22:45:17.442006 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:17.441840    4871 scope.go:122] "RemoveContainer" containerID="7ece90167ba6e39faad07fb451ce12c3b4d70d781f335b8302f2f0d3a02341bb"
Oct 30 22:45:17.442370 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:45:17.442336    4871 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w3-43d592-29890005-sxcjg_odh-ai-gateway-infra(752747f8-66a0-4631-98e8-e9207c22916f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-43d592-29890005-sxcjg" podUID="752747f8-66a0-4631-98e8-e9207c22916f"
Oct 30 22:45:17.444677 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:17.444649    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-8c852b-29890005-2vxbn" event={"ID":"fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2","Type":"ContainerStarted","Data":"8b7fcfb187c3619230533f4b7b9739181659173c8039a44046459d374ad1400a"}
Oct 30 22:45:17.509157 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:17.509128    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:17.586410 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:17.580599    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-43d592-29890005-sxcjg"]
Oct 30 22:45:18.455201 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:18.455158    4871 generic.go:356] "Generic (PLEG): container finished" podID="fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2" containerID="8b7fcfb187c3619230533f4b7b9739181659173c8039a44046459d374ad1400a" exitCode=60
Oct 30 22:45:18.455734 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:18.455231    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-8c852b-29890005-2vxbn" event={"ID":"fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2","Type":"ContainerDied","Data":"8b7fcfb187c3619230533f4b7b9739181659173c8039a44046459d374ad1400a"}
Oct 30 22:45:18.455734 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:18.455320    4871 scope.go:122] "RemoveContainer" containerID="63551e1791a3142896c506f34925027a69842ed2210dd3a7015e82625ba2b621"
Oct 30 22:45:18.455734 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:18.455710    4871 scope.go:122] "RemoveContainer" containerID="8b7fcfb187c3619230533f4b7b9739181659173c8039a44046459d374ad1400a"
Oct 30 22:45:18.456116 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:45:18.456086    4871 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-8c852b-29890005-2vxbn_odh-ai-gateway-infra(fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-8c852b-29890005-2vxbn" podUID="fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2"
Oct 30 22:45:18.457470 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:18.457444    4871 generic.go:356] "Generic (PLEG): container finished" podID="61d9910d-4558-4ade-a332-126ca461a006" containerID="b0696c890a58d5f22f7ca7abab961a7d824d69cd158815c6355985ac4fd6f30f" exitCode=60
Oct 30 22:45:18.457580 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:18.457503    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-be6ecc-29890005-wsbxj" event={"ID":"61d9910d-4558-4ade-a332-126ca461a006","Type":"ContainerDied","Data":"b0696c890a58d5f22f7ca7abab961a7d824d69cd158815c6355985ac4fd6f30f"}
Oct 30 22:45:18.457927 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:18.457907    4871 scope.go:122] "RemoveContainer" containerID="b0696c890a58d5f22f7ca7abab961a7d824d69cd158815c6355985ac4fd6f30f"
Oct 30 22:45:18.458267 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:45:18.458216    4871 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-be6ecc-29890005-wsbxj_odh-ai-gateway-infra(61d9910d-4558-4ade-a332-126ca461a006)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-be6ecc-29890005-wsbxj" podUID="61d9910d-4558-4ade-a332-126ca461a006"
Oct 30 22:45:18.469569 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:18.469542    4871 scope.go:122] "RemoveContainer" containerID="6eb23fccbe7154e7a12d3a6e561acae735a4996eba61948b7db4d2b273da861c"
Oct 30 22:45:18.665952 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:18.665898    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-43d592-29890005-sxcjg"
Oct 30 22:45:18.713365 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:18.713264    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-8c852b-29890005-2vxbn"]
Oct 30 22:45:18.742253 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:18.742202    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/752747f8-66a0-4631-98e8-e9207c22916f-kube-api-access-wxtgz\" (UniqueName: \"kubernetes.io/projected/752747f8-66a0-4631-98e8-e9207c22916f-kube-api-access-wxtgz\") pod \"752747f8-66a0-4631-98e8-e9207c22916f\" (UID: \"752747f8-66a0-4631-98e8-e9207c22916f\") "
Oct 30 22:45:18.742431 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:18.742326    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/752747f8-66a0-4631-98e8-e9207c22916f-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/752747f8-66a0-4631-98e8-e9207c22916f-maas-api-service-ca\") pod \"752747f8-66a0-4631-98e8-e9207c22916f\" (UID: \"752747f8-66a0-4631-98e8-e9207c22916f\") "
Oct 30 22:45:18.742773 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:18.742747    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/752747f8-66a0-4631-98e8-e9207c22916f-maas-api-service-ca" pod "752747f8-66a0-4631-98e8-e9207c22916f" (UID: "752747f8-66a0-4631-98e8-e9207c22916f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:45:18.742909 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:18.742892    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/752747f8-66a0-4631-98e8-e9207c22916f-maas-api-service-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:45:18.744553 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:18.744522    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/752747f8-66a0-4631-98e8-e9207c22916f-kube-api-access-wxtgz" pod "752747f8-66a0-4631-98e8-e9207c22916f" (UID: "752747f8-66a0-4631-98e8-e9207c22916f"). InnerVolumeSpecName "kube-api-access-wxtgz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:45:18.843997 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:18.843955    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxtgz\" (UniqueName: \"kubernetes.io/projected/752747f8-66a0-4631-98e8-e9207c22916f-kube-api-access-wxtgz\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:45:19.470830 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:19.470801    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-43d592-29890005-sxcjg"
Oct 30 22:45:19.471354 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:19.470804    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-43d592-29890005-sxcjg" event={"ID":"752747f8-66a0-4631-98e8-e9207c22916f","Type":"ContainerDied","Data":"9139533b8ce1f51f5442941c3262e83384d7b28d23076713834f44e8a966bafd"}
Oct 30 22:45:19.471354 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:19.470943    4871 scope.go:122] "RemoveContainer" containerID="7ece90167ba6e39faad07fb451ce12c3b4d70d781f335b8302f2f0d3a02341bb"
Oct 30 22:45:19.644042 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:19.643999    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-8c852b-29890005-2vxbn"
Oct 30 22:45:19.646273 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:19.646231    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-43d592-29890005-sxcjg"]
Oct 30 22:45:19.650834 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:19.650806    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-43d592-29890005-sxcjg"]
Oct 30 22:45:19.755385 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:19.755282    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2-kube-api-access-ppdzd\" (UniqueName: \"kubernetes.io/projected/fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2-kube-api-access-ppdzd\") pod \"fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2\" (UID: \"fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2\") "
Oct 30 22:45:19.755557 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:19.755459    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2-maas-api-service-ca\") pod \"fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2\" (UID: \"fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2\") "
Oct 30 22:45:19.756205 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:19.756159    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2-maas-api-service-ca" pod "fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2" (UID: "fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:45:19.756432 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:19.756384    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2-maas-api-service-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:45:19.757869 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:19.757837    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2-kube-api-access-ppdzd" pod "fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2" (UID: "fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2"). InnerVolumeSpecName "kube-api-access-ppdzd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:45:19.827777 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:19.827740    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-be6ecc-29890005-wsbxj"]
Oct 30 22:45:19.857915 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:19.857874    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppdzd\" (UniqueName: \"kubernetes.io/projected/fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2-kube-api-access-ppdzd\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:45:19.964926 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:19.964894    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="752747f8-66a0-4631-98e8-e9207c22916f" path="/var/lib/kubelet/pods/752747f8-66a0-4631-98e8-e9207c22916f/volumes"
Oct 30 22:45:20.021631 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:20.021547    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-be6ecc-29890005-wsbxj"
Oct 30 22:45:20.161778 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:20.161739    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/61d9910d-4558-4ade-a332-126ca461a006-kube-api-access-jp9m8\" (UniqueName: \"kubernetes.io/projected/61d9910d-4558-4ade-a332-126ca461a006-kube-api-access-jp9m8\") pod \"61d9910d-4558-4ade-a332-126ca461a006\" (UID: \"61d9910d-4558-4ade-a332-126ca461a006\") "
Oct 30 22:45:20.162007 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:20.161958    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/61d9910d-4558-4ade-a332-126ca461a006-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/61d9910d-4558-4ade-a332-126ca461a006-maas-api-service-ca\") pod \"61d9910d-4558-4ade-a332-126ca461a006\" (UID: \"61d9910d-4558-4ade-a332-126ca461a006\") "
Oct 30 22:45:20.162407 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:20.162376    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61d9910d-4558-4ade-a332-126ca461a006-maas-api-service-ca" pod "61d9910d-4558-4ade-a332-126ca461a006" (UID: "61d9910d-4558-4ade-a332-126ca461a006"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:45:20.162524 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:20.162463    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/61d9910d-4558-4ade-a332-126ca461a006-maas-api-service-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:45:20.164093 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:20.164069    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61d9910d-4558-4ade-a332-126ca461a006-kube-api-access-jp9m8" pod "61d9910d-4558-4ade-a332-126ca461a006" (UID: "61d9910d-4558-4ade-a332-126ca461a006"). InnerVolumeSpecName "kube-api-access-jp9m8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:45:20.263508 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:20.263475    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jp9m8\" (UniqueName: \"kubernetes.io/projected/61d9910d-4558-4ade-a332-126ca461a006-kube-api-access-jp9m8\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:45:20.483160 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:20.483120    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-8c852b-29890005-2vxbn"
Oct 30 22:45:20.483160 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:20.483145    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-8c852b-29890005-2vxbn" event={"ID":"fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2","Type":"ContainerDied","Data":"7a703ab101114c9d1fb73ec1c0907fbbb4beb1c51789d2fdd1b3caf0b929b17f"}
Oct 30 22:45:20.483722 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:20.483199    4871 scope.go:122] "RemoveContainer" containerID="8b7fcfb187c3619230533f4b7b9739181659173c8039a44046459d374ad1400a"
Oct 30 22:45:20.484908 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:20.484880    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-be6ecc-29890005-wsbxj" event={"ID":"61d9910d-4558-4ade-a332-126ca461a006","Type":"ContainerDied","Data":"71c2f992ac8448fea299207c16f5203c8ed48298a6c4c74f5ea16c249781ca40"}
Oct 30 22:45:20.485010 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:20.484923    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-be6ecc-29890005-wsbxj"
Oct 30 22:45:20.494504 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:20.494487    4871 scope.go:122] "RemoveContainer" containerID="b0696c890a58d5f22f7ca7abab961a7d824d69cd158815c6355985ac4fd6f30f"
Oct 30 22:45:20.659331 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:20.659292    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-be6ecc-29890005-wsbxj"]
Oct 30 22:45:20.666218 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:20.666186    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-be6ecc-29890005-wsbxj"]
Oct 30 22:45:20.785181 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:20.785085    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-8c852b-29890005-2vxbn"]
Oct 30 22:45:20.796807 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:20.796763    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-8c852b-29890005-2vxbn"]
Oct 30 22:45:21.961838 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:21.961802    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="61d9910d-4558-4ade-a332-126ca461a006" path="/var/lib/kubelet/pods/61d9910d-4558-4ade-a332-126ca461a006/volumes"
Oct 30 22:45:21.962345 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:21.962329    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2" path="/var/lib/kubelet/pods/fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2/volumes"
Oct 30 22:45:22.060548 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:22.060512    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:22.216686 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:22.216565    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:22.387043 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:22.387009    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:23.744716 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:23.744679    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:26.591563 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:26.591523    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-5a5534-c445d4f57-7vgqd"]
Oct 30 22:45:26.592175 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:26.592161    4871 cpu_manager.go:414] "containerMap: removing container" podUID="fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2" containerName="cleanup"
Oct 30 22:45:26.592220 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:26.592176    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2" containerName="cleanup"
Oct 30 22:45:26.592220 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:26.592189    4871 cpu_manager.go:414] "containerMap: removing container" podUID="fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2" containerName="cleanup"
Oct 30 22:45:26.592220 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:26.592194    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2" containerName="cleanup"
Oct 30 22:45:26.592220 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:26.592205    4871 cpu_manager.go:414] "containerMap: removing container" podUID="752747f8-66a0-4631-98e8-e9207c22916f" containerName="cleanup"
Oct 30 22:45:26.592220 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:26.592210    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="752747f8-66a0-4631-98e8-e9207c22916f" containerName="cleanup"
Oct 30 22:45:26.592408 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:26.592231    4871 cpu_manager.go:414] "containerMap: removing container" podUID="61d9910d-4558-4ade-a332-126ca461a006" containerName="cleanup"
Oct 30 22:45:26.592408 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:26.592251    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="61d9910d-4558-4ade-a332-126ca461a006" containerName="cleanup"
Oct 30 22:45:26.592408 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:26.592261    4871 cpu_manager.go:414] "containerMap: removing container" podUID="752747f8-66a0-4631-98e8-e9207c22916f" containerName="cleanup"
Oct 30 22:45:26.592408 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:26.592266    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="752747f8-66a0-4631-98e8-e9207c22916f" containerName="cleanup"
Oct 30 22:45:26.592408 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:26.592284    4871 cpu_manager.go:414] "containerMap: removing container" podUID="61d9910d-4558-4ade-a332-126ca461a006" containerName="cleanup"
Oct 30 22:45:26.592408 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:26.592289    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="61d9910d-4558-4ade-a332-126ca461a006" containerName="cleanup"
Oct 30 22:45:26.592408 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:26.592301    4871 cpu_manager.go:414] "containerMap: removing container" podUID="61d9910d-4558-4ade-a332-126ca461a006" containerName="cleanup"
Oct 30 22:45:26.592408 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:26.592311    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="61d9910d-4558-4ade-a332-126ca461a006" containerName="cleanup"
Oct 30 22:45:26.592648 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:26.592423    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="752747f8-66a0-4631-98e8-e9207c22916f" containerName="cleanup"
Oct 30 22:45:26.592648 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:26.592433    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="61d9910d-4558-4ade-a332-126ca461a006" containerName="cleanup"
Oct 30 22:45:26.592648 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:26.592441    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2" containerName="cleanup"
Oct 30 22:45:26.592648 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:26.592448    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="61d9910d-4558-4ade-a332-126ca461a006" containerName="cleanup"
Oct 30 22:45:26.592648 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:26.592453    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="61d9910d-4558-4ade-a332-126ca461a006" containerName="cleanup"
Oct 30 22:45:26.592648 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:26.592458    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="752747f8-66a0-4631-98e8-e9207c22916f" containerName="cleanup"
Oct 30 22:45:26.592648 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:26.592466    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2" containerName="cleanup"
Oct 30 22:45:26.597760 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:26.597738    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5534-c445d4f57-7vgqd"
Oct 30 22:45:26.653617 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:26.653576    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-5a5534-c445d4f57-7vgqd"]
Oct 30 22:45:26.699307 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:26.699274    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-5a5534-serving-cert\""
Oct 30 22:45:26.753672 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:26.753628    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a874ed5-421c-4199-87bf-ed1997ceb64b-openshift-service-ca\") pod \"maas-api-e2e-autores-5a5534-c445d4f57-7vgqd\" (UID: \"0a874ed5-421c-4199-87bf-ed1997ceb64b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5534-c445d4f57-7vgqd"
Oct 30 22:45:26.753900 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:26.753794    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a874ed5-421c-4199-87bf-ed1997ceb64b-maas-api-tls\") pod \"maas-api-e2e-autores-5a5534-c445d4f57-7vgqd\" (UID: \"0a874ed5-421c-4199-87bf-ed1997ceb64b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5534-c445d4f57-7vgqd"
Oct 30 22:45:26.753900 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:26.753836    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a874ed5-421c-4199-87bf-ed1997ceb64b-metrics-certs\") pod \"maas-api-e2e-autores-5a5534-c445d4f57-7vgqd\" (UID: \"0a874ed5-421c-4199-87bf-ed1997ceb64b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5534-c445d4f57-7vgqd"
Oct 30 22:45:26.754027 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:26.754009    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48skg\" (UniqueName: \"kubernetes.io/projected/0a874ed5-421c-4199-87bf-ed1997ceb64b-kube-api-access-48skg\") pod \"maas-api-e2e-autores-5a5534-c445d4f57-7vgqd\" (UID: \"0a874ed5-421c-4199-87bf-ed1997ceb64b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5534-c445d4f57-7vgqd"
Oct 30 22:45:26.780479 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:26.780447    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:26.870788 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:26.870748    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a874ed5-421c-4199-87bf-ed1997ceb64b-maas-api-tls\") pod \"maas-api-e2e-autores-5a5534-c445d4f57-7vgqd\" (UID: \"0a874ed5-421c-4199-87bf-ed1997ceb64b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5534-c445d4f57-7vgqd"
Oct 30 22:45:26.872568 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:26.872538    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a874ed5-421c-4199-87bf-ed1997ceb64b-metrics-certs\") pod \"maas-api-e2e-autores-5a5534-c445d4f57-7vgqd\" (UID: \"0a874ed5-421c-4199-87bf-ed1997ceb64b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5534-c445d4f57-7vgqd"
Oct 30 22:45:26.887358 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:26.887321    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48skg\" (UniqueName: \"kubernetes.io/projected/0a874ed5-421c-4199-87bf-ed1997ceb64b-kube-api-access-48skg\") pod \"maas-api-e2e-autores-5a5534-c445d4f57-7vgqd\" (UID: \"0a874ed5-421c-4199-87bf-ed1997ceb64b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5534-c445d4f57-7vgqd"
Oct 30 22:45:26.888351 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:26.888322    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a874ed5-421c-4199-87bf-ed1997ceb64b-openshift-service-ca\") pod \"maas-api-e2e-autores-5a5534-c445d4f57-7vgqd\" (UID: \"0a874ed5-421c-4199-87bf-ed1997ceb64b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5534-c445d4f57-7vgqd"
Oct 30 22:45:26.898374 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:26.889216    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a874ed5-421c-4199-87bf-ed1997ceb64b-openshift-service-ca\") pod \"maas-api-e2e-autores-5a5534-c445d4f57-7vgqd\" (UID: \"0a874ed5-421c-4199-87bf-ed1997ceb64b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5534-c445d4f57-7vgqd"
Oct 30 22:45:26.898639 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:26.897852    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a874ed5-421c-4199-87bf-ed1997ceb64b-metrics-certs\") pod \"maas-api-e2e-autores-5a5534-c445d4f57-7vgqd\" (UID: \"0a874ed5-421c-4199-87bf-ed1997ceb64b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5534-c445d4f57-7vgqd"
Oct 30 22:45:26.899572 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:26.899545    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a874ed5-421c-4199-87bf-ed1997ceb64b-maas-api-tls\") pod \"maas-api-e2e-autores-5a5534-c445d4f57-7vgqd\" (UID: \"0a874ed5-421c-4199-87bf-ed1997ceb64b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5534-c445d4f57-7vgqd"
Oct 30 22:45:26.971718 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:26.971679    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-48skg\" (UniqueName: \"kubernetes.io/projected/0a874ed5-421c-4199-87bf-ed1997ceb64b-kube-api-access-48skg\") pod \"maas-api-e2e-autores-5a5534-c445d4f57-7vgqd\" (UID: \"0a874ed5-421c-4199-87bf-ed1997ceb64b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5534-c445d4f57-7vgqd"
Oct 30 22:45:27.054395 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:27.054363    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:27.213224 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:27.213124    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5534-c445d4f57-7vgqd"
Oct 30 22:45:27.225046 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:27.225012    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:27.607558 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:45:27.607520    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a874ed5_421c_4199_87bf_ed1997ceb64b.slice/crio-7e7fb36cc3b1152f654e030259cfe70e90c84cdf944780a20e86747dc8363856 WatchSource:0}: Error finding container 7e7fb36cc3b1152f654e030259cfe70e90c84cdf944780a20e86747dc8363856: Status 404 returned error can't find the container with id 7e7fb36cc3b1152f654e030259cfe70e90c84cdf944780a20e86747dc8363856
Oct 30 22:45:27.614074 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:27.614036    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-5a5534-c445d4f57-7vgqd"]
Oct 30 22:45:28.562711 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:28.562673    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5534-c445d4f57-7vgqd" event={"ID":"0a874ed5-421c-4199-87bf-ed1997ceb64b","Type":"ContainerStarted","Data":"7e7fb36cc3b1152f654e030259cfe70e90c84cdf944780a20e86747dc8363856"}
Oct 30 22:45:29.624215 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:29.624173    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5534-c445d4f57-7vgqd" event={"ID":"0a874ed5-421c-4199-87bf-ed1997ceb64b","Type":"ContainerStarted","Data":"1ff27fc9fbe6cc3b03910c5e0c39048a22ec4cb7de60a56d90429304b0fb1f62"}
Oct 30 22:45:29.626270 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:29.626224    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5534-c445d4f57-7vgqd"
Oct 30 22:45:29.755769 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:29.755671    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5534-c445d4f57-7vgqd" podStartSLOduration=2.788037138 podStartE2EDuration="3.755652473s" totalImagesPullingTime="967.615335ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:45:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:45:29.754689101 +0000 UTC m=+4071.932177719" watchObservedRunningTime="2026-10-30 22:45:29.755652473 +0000 UTC m=+4071.933141091"
Oct 30 22:45:31.612032 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:31.611990    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:33.129226 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:33.129186    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:35.568750 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:45:35.568700    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:45:36.208911 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:36.208855    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:36.687137 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:36.687107    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5534-c445d4f57-7vgqd"
Oct 30 22:45:40.797544 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:40.797503    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:45.327010 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:45.326968    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:49.861942 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:49.861852    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:52.322604 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:52.322572    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:52.533097 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:52.533064    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:53.756340 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:53.756302    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:54.565867 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:54.564831    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:54.737069 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:54.737023    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:54.900313 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:54.900280    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:59.482647 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:45:59.482617    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:46:04.109698 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:04.109660    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:46:05.570249 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:46:05.570192    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:46:06.973432 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:06.973395    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:46:07.137230 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:07.137199    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:46:08.108965 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:08.108920    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:46:09.888013 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:09.887965    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:46:11.395710 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:11.395679    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:46:11.691563 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:11.691480    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:46:14.698858 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:14.698812    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:46:14.850802 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:14.850746    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:46:15.066338 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:15.066229    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:46:19.606618 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:19.597609    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:46:22.446682 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:22.446647    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:46:24.409249 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:24.409204    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:46:26.209498 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.209447    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"]
Oct 30 22:46:26.210159 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.210143    4871 cpu_manager.go:414] "containerMap: removing container" podUID="752747f8-66a0-4631-98e8-e9207c22916f" containerName="cleanup"
Oct 30 22:46:26.210159 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.210158    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="752747f8-66a0-4631-98e8-e9207c22916f" containerName="cleanup"
Oct 30 22:46:26.210330 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.210194    4871 cpu_manager.go:414] "containerMap: removing container" podUID="fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2" containerName="cleanup"
Oct 30 22:46:26.210330 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.210200    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2" containerName="cleanup"
Oct 30 22:46:26.210455 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.210346    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="752747f8-66a0-4631-98e8-e9207c22916f" containerName="cleanup"
Oct 30 22:46:26.210455 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.210361    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="fa7d03a2-a0fd-4fb2-87be-6eacc1a672f2" containerName="cleanup"
Oct 30 22:46:26.214333 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.214308    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"
Oct 30 22:46:26.269010 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.268972    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-6e0b99-openshift-default-dockercfg-pcv5l\""
Oct 30 22:46:26.286902 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.286865    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghxjb\" (UniqueName: \"kubernetes.io/projected/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-kube-api-access-ghxjb\") pod \"e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd\" (UID: \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\") " pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"
Oct 30 22:46:26.287113 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.286928    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-istiod-ca-cert\") pod \"e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd\" (UID: \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\") " pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"
Oct 30 22:46:26.287113 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.287010    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-istio-envoy\") pod \"e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd\" (UID: \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\") " pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"
Oct 30 22:46:26.287288 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.287147    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-istio-podinfo\") pod \"e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd\" (UID: \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\") " pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"
Oct 30 22:46:26.287288 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.287277    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-workload-socket\") pod \"e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd\" (UID: \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\") " pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"
Oct 30 22:46:26.287394 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.287345    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-istio-token\") pod \"e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd\" (UID: \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\") " pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"
Oct 30 22:46:26.287439 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.287411    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-workload-certs\") pod \"e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd\" (UID: \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\") " pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"
Oct 30 22:46:26.287509 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.287491    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-credential-socket\") pod \"e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd\" (UID: \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\") " pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"
Oct 30 22:46:26.287559 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.287529    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-istio-data\") pod \"e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd\" (UID: \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\") " pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"
Oct 30 22:46:26.328196 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.328157    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"]
Oct 30 22:46:26.388737 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.388701    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-istio-podinfo\") pod \"e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd\" (UID: \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\") " pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"
Oct 30 22:46:26.388944 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.388789    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-workload-socket\") pod \"e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd\" (UID: \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\") " pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"
Oct 30 22:46:26.388944 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.388831    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-istio-token\") pod \"e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd\" (UID: \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\") " pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"
Oct 30 22:46:26.388944 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.388852    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-workload-certs\") pod \"e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd\" (UID: \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\") " pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"
Oct 30 22:46:26.388944 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.388898    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-credential-socket\") pod \"e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd\" (UID: \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\") " pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"
Oct 30 22:46:26.389285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.388949    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-istio-data\") pod \"e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd\" (UID: \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\") " pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"
Oct 30 22:46:26.389285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.389155    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghxjb\" (UniqueName: \"kubernetes.io/projected/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-kube-api-access-ghxjb\") pod \"e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd\" (UID: \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\") " pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"
Oct 30 22:46:26.389421 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.389288    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-istiod-ca-cert\") pod \"e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd\" (UID: \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\") " pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"
Oct 30 22:46:26.389421 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.389316    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-workload-socket\") pod \"e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd\" (UID: \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\") " pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"
Oct 30 22:46:26.389421 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.389337    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-workload-certs\") pod \"e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd\" (UID: \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\") " pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"
Oct 30 22:46:26.389421 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.389323    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-istio-envoy\") pod \"e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd\" (UID: \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\") " pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"
Oct 30 22:46:26.389421 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.389404    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-credential-socket\") pod \"e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd\" (UID: \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\") " pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"
Oct 30 22:46:26.389622 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.389426    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-istio-data\") pod \"e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd\" (UID: \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\") " pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"
Oct 30 22:46:26.389899 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.389882    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-istiod-ca-cert\") pod \"e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd\" (UID: \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\") " pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"
Oct 30 22:46:26.391300 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.391278    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-istio-envoy\") pod \"e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd\" (UID: \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\") " pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"
Oct 30 22:46:26.391392 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.391371    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-istio-podinfo\") pod \"e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd\" (UID: \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\") " pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"
Oct 30 22:46:26.453299 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.453259    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-istio-token\") pod \"e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd\" (UID: \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\") " pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"
Oct 30 22:46:26.453501 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.453323    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghxjb\" (UniqueName: \"kubernetes.io/projected/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-kube-api-access-ghxjb\") pod \"e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd\" (UID: \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\") " pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"
Oct 30 22:46:26.541419 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.541316    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"
Oct 30 22:46:26.983585 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:46:26.983548    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e7a9233_fd96_4bf4_8f1e_3b9b84e02a2e.slice/crio-b69af78e14c8e1a382cfb7c2d5ed9fc0d71fec3b15862d711c0b45bee9adf687 WatchSource:0}: Error finding container b69af78e14c8e1a382cfb7c2d5ed9fc0d71fec3b15862d711c0b45bee9adf687: Status 404 returned error can't find the container with id b69af78e14c8e1a382cfb7c2d5ed9fc0d71fec3b15862d711c0b45bee9adf687
Oct 30 22:46:26.986420 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.986386    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:46:26.986521 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.986456    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:46:26.986521 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.986484    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:46:26.986988 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:26.986965    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"]
Oct 30 22:46:27.156316 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:27.156273    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd" event={"ID":"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e","Type":"ContainerStarted","Data":"c7a9239fdb8d99e476eda85a3137ed07f23598c4dac44df92daa04806df1045e"}
Oct 30 22:46:27.156471 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:27.156333    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd" event={"ID":"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e","Type":"ContainerStarted","Data":"b69af78e14c8e1a382cfb7c2d5ed9fc0d71fec3b15862d711c0b45bee9adf687"}
Oct 30 22:46:27.284140 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:27.284021    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd" podStartSLOduration=1.284002877 podStartE2EDuration="1.284002877s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:46:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:46:27.282749252 +0000 UTC m=+4129.460237868" watchObservedRunningTime="2026-10-30 22:46:27.284002877 +0000 UTC m=+4129.461491552"
Oct 30 22:46:27.541854 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:27.541764    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"
Oct 30 22:46:28.543481 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:28.543440    4871 patch_prober.go:29] interesting pod/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.202:15021/healthz/ready\": dial tcp 10.133.0.202:15021: connect: connection refused" start-of-body=
Oct 30 22:46:28.543985 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:28.543536    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd" podUID="4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.202:15021/healthz/ready\": dial tcp 10.133.0.202:15021: connect: connection refused"
Oct 30 22:46:28.839464 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:28.839370    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:46:29.020564 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:29.020528    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:46:29.181213 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:29.181162    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:46:29.227262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:29.203137    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"]
Oct 30 22:46:29.239263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:29.227763    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr" podUID="1cb08cf4-430f-416a-b1f8-81b5695e3f4e" containerName="istio-proxy" containerID="cri-o://12dc24aea3a707a18fc5229f060f96992fef4705a4d148eef6e3d6fcb7c5b08b" gracePeriod=30
Oct 30 22:46:29.337260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:29.336723    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:46:29.541888 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:29.541795    4871 patch_prober.go:29] interesting pod/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.202:15021/healthz/ready\": dial tcp 10.133.0.202:15021: connect: connection refused" start-of-body=
Oct 30 22:46:29.542172 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:29.542143    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd" podUID="4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.202:15021/healthz/ready\": dial tcp 10.133.0.202:15021: connect: connection refused"
Oct 30 22:46:30.542804 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:30.542753    4871 patch_prober.go:29] interesting pod/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.202:15021/healthz/ready\": dial tcp 10.133.0.202:15021: connect: connection refused" start-of-body=
Oct 30 22:46:30.543251 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:30.542819    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd" podUID="4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.202:15021/healthz/ready\": dial tcp 10.133.0.202:15021: connect: connection refused"
Oct 30 22:46:31.542385 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:31.542345    4871 patch_prober.go:29] interesting pod/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.202:15021/healthz/ready\": dial tcp 10.133.0.202:15021: connect: connection refused" start-of-body=
Oct 30 22:46:31.542594 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:31.542428    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd" podUID="4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.202:15021/healthz/ready\": dial tcp 10.133.0.202:15021: connect: connection refused"
Oct 30 22:46:32.541964 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:32.541928    4871 patch_prober.go:29] interesting pod/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.202:15021/healthz/ready\": dial tcp 10.133.0.202:15021: connect: connection refused" start-of-body=
Oct 30 22:46:32.542507 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:32.541985    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd" podUID="4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.202:15021/healthz/ready\": dial tcp 10.133.0.202:15021: connect: connection refused"
Oct 30 22:46:33.547132 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:33.547101    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"
Oct 30 22:46:33.547675 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:33.547484    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"
Oct 30 22:46:33.548229 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:33.548207    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"
Oct 30 22:46:33.957828 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:33.957794    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:46:34.532829 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:34.532799    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"
Oct 30 22:46:34.668561 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:34.668499    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-workload-certs\") pod \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\" (UID: \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\") "
Oct 30 22:46:34.668561 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:34.668567    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-istio-data\") pod \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\" (UID: \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\") "
Oct 30 22:46:34.669133 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:34.668679    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-workload-socket\") pod \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\" (UID: \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\") "
Oct 30 22:46:34.669133 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:34.668836    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-workload-certs" pod "1cb08cf4-430f-416a-b1f8-81b5695e3f4e" (UID: "1cb08cf4-430f-416a-b1f8-81b5695e3f4e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:46:34.669133 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:34.668842    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-istio-data" pod "1cb08cf4-430f-416a-b1f8-81b5695e3f4e" (UID: "1cb08cf4-430f-416a-b1f8-81b5695e3f4e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:46:34.669133 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:34.668922    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-kube-api-access-tqzhf\" (UniqueName: \"kubernetes.io/projected/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-kube-api-access-tqzhf\") pod \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\" (UID: \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\") "
Oct 30 22:46:34.669133 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:34.668932    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-workload-socket" pod "1cb08cf4-430f-416a-b1f8-81b5695e3f4e" (UID: "1cb08cf4-430f-416a-b1f8-81b5695e3f4e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:46:34.669133 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:34.668966    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-istio-token\" (UniqueName: \"kubernetes.io/projected/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-istio-token\") pod \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\" (UID: \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\") "
Oct 30 22:46:34.669133 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:34.669000    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-istio-envoy\") pod \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\" (UID: \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\") "
Oct 30 22:46:34.669133 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:34.669035    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-istiod-ca-cert\") pod \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\" (UID: \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\") "
Oct 30 22:46:34.669133 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:34.669077    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-istio-podinfo\") pod \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\" (UID: \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\") "
Oct 30 22:46:34.669653 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:34.669142    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-credential-socket\") pod \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\" (UID: \"1cb08cf4-430f-416a-b1f8-81b5695e3f4e\") "
Oct 30 22:46:34.669653 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:34.669476    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-credential-socket" pod "1cb08cf4-430f-416a-b1f8-81b5695e3f4e" (UID: "1cb08cf4-430f-416a-b1f8-81b5695e3f4e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:46:34.669653 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:34.669493    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-istiod-ca-cert" pod "1cb08cf4-430f-416a-b1f8-81b5695e3f4e" (UID: "1cb08cf4-430f-416a-b1f8-81b5695e3f4e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:46:34.670053 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:34.670033    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:46:34.670144 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:34.670057    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:46:34.670144 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:34.670071    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:46:34.670144 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:34.670084    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:46:34.670144 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:34.670099    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:46:34.671337 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:34.671311    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-kube-api-access-tqzhf" pod "1cb08cf4-430f-416a-b1f8-81b5695e3f4e" (UID: "1cb08cf4-430f-416a-b1f8-81b5695e3f4e"). InnerVolumeSpecName "kube-api-access-tqzhf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:46:34.671432 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:34.671356    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-istio-token" pod "1cb08cf4-430f-416a-b1f8-81b5695e3f4e" (UID: "1cb08cf4-430f-416a-b1f8-81b5695e3f4e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:46:34.671634 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:34.671611    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-istio-envoy" pod "1cb08cf4-430f-416a-b1f8-81b5695e3f4e" (UID: "1cb08cf4-430f-416a-b1f8-81b5695e3f4e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:46:34.671736 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:34.671645    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-istio-podinfo" pod "1cb08cf4-430f-416a-b1f8-81b5695e3f4e" (UID: "1cb08cf4-430f-416a-b1f8-81b5695e3f4e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:46:34.771739 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:34.771651    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqzhf\" (UniqueName: \"kubernetes.io/projected/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-kube-api-access-tqzhf\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:46:34.771739 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:34.771686    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:46:34.771739 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:34.771699    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:46:34.771739 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:34.771707    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cb08cf4-430f-416a-b1f8-81b5695e3f4e-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:46:35.249843 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:35.249805    4871 generic.go:356] "Generic (PLEG): container finished" podID="1cb08cf4-430f-416a-b1f8-81b5695e3f4e" containerID="12dc24aea3a707a18fc5229f060f96992fef4705a4d148eef6e3d6fcb7c5b08b" exitCode=0
Oct 30 22:46:35.250043 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:35.249862    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"
Oct 30 22:46:35.250043 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:35.249896    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr" event={"ID":"1cb08cf4-430f-416a-b1f8-81b5695e3f4e","Type":"ContainerDied","Data":"12dc24aea3a707a18fc5229f060f96992fef4705a4d148eef6e3d6fcb7c5b08b"}
Oct 30 22:46:35.250043 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:35.249953    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr" event={"ID":"1cb08cf4-430f-416a-b1f8-81b5695e3f4e","Type":"ContainerDied","Data":"b6ad760178354114f16c353c135ada3f7f0ae6290beabd01cc741215cb5731c1"}
Oct 30 22:46:35.250043 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:35.249982    4871 scope.go:122] "RemoveContainer" containerID="12dc24aea3a707a18fc5229f060f96992fef4705a4d148eef6e3d6fcb7c5b08b"
Oct 30 22:46:35.263825 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:35.263794    4871 scope.go:122] "RemoveContainer" containerID="6d5103abf164ff54e5de46fdcd830550176dfbb336a4a6162e4e0f4327ea439c"
Oct 30 22:46:35.275846 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:35.275817    4871 scope.go:122] "RemoveContainer" containerID="12dc24aea3a707a18fc5229f060f96992fef4705a4d148eef6e3d6fcb7c5b08b"
Oct 30 22:46:35.276165 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:46:35.276142    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12dc24aea3a707a18fc5229f060f96992fef4705a4d148eef6e3d6fcb7c5b08b\": container with ID starting with 12dc24aea3a707a18fc5229f060f96992fef4705a4d148eef6e3d6fcb7c5b08b not found: ID does not exist" containerID="12dc24aea3a707a18fc5229f060f96992fef4705a4d148eef6e3d6fcb7c5b08b"
Oct 30 22:46:35.276230 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:35.276175    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12dc24aea3a707a18fc5229f060f96992fef4705a4d148eef6e3d6fcb7c5b08b"} err="failed to get container status \"12dc24aea3a707a18fc5229f060f96992fef4705a4d148eef6e3d6fcb7c5b08b\": rpc error: code = NotFound desc = could not find container \"12dc24aea3a707a18fc5229f060f96992fef4705a4d148eef6e3d6fcb7c5b08b\": container with ID starting with 12dc24aea3a707a18fc5229f060f96992fef4705a4d148eef6e3d6fcb7c5b08b not found: ID does not exist"
Oct 30 22:46:35.276230 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:35.276198    4871 scope.go:122] "RemoveContainer" containerID="6d5103abf164ff54e5de46fdcd830550176dfbb336a4a6162e4e0f4327ea439c"
Oct 30 22:46:35.276538 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:46:35.276518    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d5103abf164ff54e5de46fdcd830550176dfbb336a4a6162e4e0f4327ea439c\": container with ID starting with 6d5103abf164ff54e5de46fdcd830550176dfbb336a4a6162e4e0f4327ea439c not found: ID does not exist" containerID="6d5103abf164ff54e5de46fdcd830550176dfbb336a4a6162e4e0f4327ea439c"
Oct 30 22:46:35.276581 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:35.276547    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d5103abf164ff54e5de46fdcd830550176dfbb336a4a6162e4e0f4327ea439c"} err="failed to get container status \"6d5103abf164ff54e5de46fdcd830550176dfbb336a4a6162e4e0f4327ea439c\": rpc error: code = NotFound desc = could not find container \"6d5103abf164ff54e5de46fdcd830550176dfbb336a4a6162e4e0f4327ea439c\": container with ID starting with 6d5103abf164ff54e5de46fdcd830550176dfbb336a4a6162e4e0f4327ea439c not found: ID does not exist"
Oct 30 22:46:35.477140 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:35.477109    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"]
Oct 30 22:46:35.529197 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:35.529109    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-be6ecc-openshift-default-59846f5749-bvpjr"]
Oct 30 22:46:35.569805 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:46:35.569767    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:46:35.969369 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:35.969335    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1cb08cf4-430f-416a-b1f8-81b5695e3f4e" path="/var/lib/kubelet/pods/1cb08cf4-430f-416a-b1f8-81b5695e3f4e/volumes"
Oct 30 22:46:38.880820 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:38.880789    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:46:43.726631 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:43.726592    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:46:44.191282 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:44.191247    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:46:44.346862 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:44.346831    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:46:48.766820 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:48.766770    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:46:52.651422 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:52.651378    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:46:53.657953 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:53.657880    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:46:54.114479 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:54.114447    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:46:58.755476 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:46:58.755420    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:00.934606 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:00.934564    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:01.140122 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:01.139209    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:03.306735 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:03.306695    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:03.466564 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:03.466531    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:03.614106 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:03.614070    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:05.007076 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:05.007035    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:05.205755 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:05.205714    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:05.572181 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:47:05.572138    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:47:07.618975 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:07.618941    4871 generic.go:356] "Generic (PLEG): container finished" podID="38bc129a-15c0-4019-b32d-e4b62e79b513" containerID="b94fbf535e2488f0a0a0edf651ee20c0a96d29d6a32132410586abaeaddf90d1" exitCode=0
Oct 30 22:47:07.619496 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:07.619012    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm" event={"ID":"38bc129a-15c0-4019-b32d-e4b62e79b513","Type":"ContainerDied","Data":"b94fbf535e2488f0a0a0edf651ee20c0a96d29d6a32132410586abaeaddf90d1"}
Oct 30 22:47:07.619545 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:07.619503    4871 scope.go:122] "RemoveContainer" containerID="b94fbf535e2488f0a0a0edf651ee20c0a96d29d6a32132410586abaeaddf90d1"
Oct 30 22:47:07.621359 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:07.621329    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:47:07.621425 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:07.621407    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:47:07.621475 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:07.621442    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:47:08.101101 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:08.101011    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:08.637766 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:08.637722    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm" event={"ID":"38bc129a-15c0-4019-b32d-e4b62e79b513","Type":"ContainerStarted","Data":"6fe5f2325e0c3a0a3e1f738b5e4abddc17cc27e8f7dad3e9a579f116b7455739"}
Oct 30 22:47:08.995983 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:08.995903    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"
Oct 30 22:47:08.996167 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:08.996104    4871 patch_prober.go:29] interesting pod/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.197:15021/healthz/ready\": dial tcp 10.133.0.197:15021: connect: connection refused" start-of-body=
Oct 30 22:47:08.996263 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:08.996162    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm" podUID="38bc129a-15c0-4019-b32d-e4b62e79b513" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.197:15021/healthz/ready\": dial tcp 10.133.0.197:15021: connect: connection refused"
Oct 30 22:47:09.996414 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:09.996375    4871 patch_prober.go:29] interesting pod/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.197:15021/healthz/ready\": dial tcp 10.133.0.197:15021: connect: connection refused" start-of-body=
Oct 30 22:47:09.996912 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:09.996432    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm" podUID="38bc129a-15c0-4019-b32d-e4b62e79b513" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.197:15021/healthz/ready\": dial tcp 10.133.0.197:15021: connect: connection refused"
Oct 30 22:47:10.995933 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:10.995895    4871 patch_prober.go:29] interesting pod/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.197:15021/healthz/ready\": dial tcp 10.133.0.197:15021: connect: connection refused" start-of-body=
Oct 30 22:47:10.996142 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:10.995979    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm" podUID="38bc129a-15c0-4019-b32d-e4b62e79b513" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.197:15021/healthz/ready\": dial tcp 10.133.0.197:15021: connect: connection refused"
Oct 30 22:47:11.996525 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:11.996490    4871 patch_prober.go:29] interesting pod/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.197:15021/healthz/ready\": dial tcp 10.133.0.197:15021: connect: connection refused" start-of-body=
Oct 30 22:47:11.996953 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:11.996555    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm" podUID="38bc129a-15c0-4019-b32d-e4b62e79b513" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.197:15021/healthz/ready\": dial tcp 10.133.0.197:15021: connect: connection refused"
Oct 30 22:47:12.119271 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:12.113479    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-5a5534-c445d4f57-7vgqd"]
Oct 30 22:47:12.119781 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:12.119745    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5534-c445d4f57-7vgqd" podUID="0a874ed5-421c-4199-87bf-ed1997ceb64b" containerName="maas-api" containerID="cri-o://1ff27fc9fbe6cc3b03910c5e0c39048a22ec4cb7de60a56d90429304b0fb1f62" gracePeriod=30
Oct 30 22:47:12.278410 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:12.276937    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:12.685536 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:12.685497    4871 generic.go:356] "Generic (PLEG): container finished" podID="0a874ed5-421c-4199-87bf-ed1997ceb64b" containerID="1ff27fc9fbe6cc3b03910c5e0c39048a22ec4cb7de60a56d90429304b0fb1f62" exitCode=0
Oct 30 22:47:12.685727 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:12.685595    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5534-c445d4f57-7vgqd" event={"ID":"0a874ed5-421c-4199-87bf-ed1997ceb64b","Type":"ContainerDied","Data":"1ff27fc9fbe6cc3b03910c5e0c39048a22ec4cb7de60a56d90429304b0fb1f62"}
Oct 30 22:47:12.783540 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:12.783506    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5534-c445d4f57-7vgqd"
Oct 30 22:47:12.938259 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:12.935319    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0a874ed5-421c-4199-87bf-ed1997ceb64b-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a874ed5-421c-4199-87bf-ed1997ceb64b-maas-api-tls\") pod \"0a874ed5-421c-4199-87bf-ed1997ceb64b\" (UID: \"0a874ed5-421c-4199-87bf-ed1997ceb64b\") "
Oct 30 22:47:12.938259 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:12.935399    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0a874ed5-421c-4199-87bf-ed1997ceb64b-kube-api-access-48skg\" (UniqueName: \"kubernetes.io/projected/0a874ed5-421c-4199-87bf-ed1997ceb64b-kube-api-access-48skg\") pod \"0a874ed5-421c-4199-87bf-ed1997ceb64b\" (UID: \"0a874ed5-421c-4199-87bf-ed1997ceb64b\") "
Oct 30 22:47:12.938259 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:12.935452    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0a874ed5-421c-4199-87bf-ed1997ceb64b-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a874ed5-421c-4199-87bf-ed1997ceb64b-openshift-service-ca\") pod \"0a874ed5-421c-4199-87bf-ed1997ceb64b\" (UID: \"0a874ed5-421c-4199-87bf-ed1997ceb64b\") "
Oct 30 22:47:12.938259 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:12.935521    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0a874ed5-421c-4199-87bf-ed1997ceb64b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a874ed5-421c-4199-87bf-ed1997ceb64b-metrics-certs\") pod \"0a874ed5-421c-4199-87bf-ed1997ceb64b\" (UID: \"0a874ed5-421c-4199-87bf-ed1997ceb64b\") "
Oct 30 22:47:12.938259 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:12.938180    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a874ed5-421c-4199-87bf-ed1997ceb64b-openshift-service-ca" pod "0a874ed5-421c-4199-87bf-ed1997ceb64b" (UID: "0a874ed5-421c-4199-87bf-ed1997ceb64b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:47:12.945884 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:12.944499    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a874ed5-421c-4199-87bf-ed1997ceb64b-kube-api-access-48skg" pod "0a874ed5-421c-4199-87bf-ed1997ceb64b" (UID: "0a874ed5-421c-4199-87bf-ed1997ceb64b"). InnerVolumeSpecName "kube-api-access-48skg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:47:12.947226 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:12.947182    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a874ed5-421c-4199-87bf-ed1997ceb64b-metrics-certs" pod "0a874ed5-421c-4199-87bf-ed1997ceb64b" (UID: "0a874ed5-421c-4199-87bf-ed1997ceb64b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:47:12.947494 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:12.947434    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a874ed5-421c-4199-87bf-ed1997ceb64b-maas-api-tls" pod "0a874ed5-421c-4199-87bf-ed1997ceb64b" (UID: "0a874ed5-421c-4199-87bf-ed1997ceb64b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:47:12.996207 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:12.996167    4871 patch_prober.go:29] interesting pod/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.197:15021/healthz/ready\": dial tcp 10.133.0.197:15021: connect: connection refused" start-of-body=
Oct 30 22:47:12.996389 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:12.996224    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm" podUID="38bc129a-15c0-4019-b32d-e4b62e79b513" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.197:15021/healthz/ready\": dial tcp 10.133.0.197:15021: connect: connection refused"
Oct 30 22:47:13.045866 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:13.045749    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a874ed5-421c-4199-87bf-ed1997ceb64b-maas-api-tls\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:47:13.045866 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:13.045789    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48skg\" (UniqueName: \"kubernetes.io/projected/0a874ed5-421c-4199-87bf-ed1997ceb64b-kube-api-access-48skg\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:47:13.045866 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:13.045806    4871 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a874ed5-421c-4199-87bf-ed1997ceb64b-openshift-service-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:47:13.045866 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:13.045820    4871 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a874ed5-421c-4199-87bf-ed1997ceb64b-metrics-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:47:13.695409 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:13.695375    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5534-c445d4f57-7vgqd"
Oct 30 22:47:13.695630 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:13.695508    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5534-c445d4f57-7vgqd" event={"ID":"0a874ed5-421c-4199-87bf-ed1997ceb64b","Type":"ContainerDied","Data":"7e7fb36cc3b1152f654e030259cfe70e90c84cdf944780a20e86747dc8363856"}
Oct 30 22:47:13.695630 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:13.695558    4871 scope.go:122] "RemoveContainer" containerID="1ff27fc9fbe6cc3b03910c5e0c39048a22ec4cb7de60a56d90429304b0fb1f62"
Oct 30 22:47:13.882712 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:13.882673    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-5a5534-c445d4f57-7vgqd"]
Oct 30 22:47:13.889135 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:13.889105    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-5a5534-c445d4f57-7vgqd"]
Oct 30 22:47:13.941960 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:13.941909    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:13.965791 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:13.965713    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0a874ed5-421c-4199-87bf-ed1997ceb64b" path="/var/lib/kubelet/pods/0a874ed5-421c-4199-87bf-ed1997ceb64b/volumes"
Oct 30 22:47:14.052279 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:14.052228    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"
Oct 30 22:47:14.053362 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:14.053323    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"
Oct 30 22:47:14.056100 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:14.056058    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"
Oct 30 22:47:14.290291 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:14.290176    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:14.457604 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:14.457565    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:18.630670 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:18.628930    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2"]
Oct 30 22:47:18.630670 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:18.629278    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2" podUID="533efba5-8b6c-4726-b0eb-2f7b53ac5ea3" containerName="maas-api" containerID="cri-o://6c4e45948ea85a178300e7028108424dfe1a30bcf241ee98312a2892071edaca" gracePeriod=30
Oct 30 22:47:18.750942 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:18.750906    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:18.956381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:18.956356    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2"
Oct 30 22:47:19.033266 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:19.032704    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/533efba5-8b6c-4726-b0eb-2f7b53ac5ea3-metrics-certs\" (UniqueName: \"kubernetes.io/secret/533efba5-8b6c-4726-b0eb-2f7b53ac5ea3-metrics-certs\") pod \"533efba5-8b6c-4726-b0eb-2f7b53ac5ea3\" (UID: \"533efba5-8b6c-4726-b0eb-2f7b53ac5ea3\") "
Oct 30 22:47:19.033266 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:19.032945    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/533efba5-8b6c-4726-b0eb-2f7b53ac5ea3-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/533efba5-8b6c-4726-b0eb-2f7b53ac5ea3-openshift-service-ca\") pod \"533efba5-8b6c-4726-b0eb-2f7b53ac5ea3\" (UID: \"533efba5-8b6c-4726-b0eb-2f7b53ac5ea3\") "
Oct 30 22:47:19.033266 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:19.033013    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/533efba5-8b6c-4726-b0eb-2f7b53ac5ea3-kube-api-access-2xsrk\" (UniqueName: \"kubernetes.io/projected/533efba5-8b6c-4726-b0eb-2f7b53ac5ea3-kube-api-access-2xsrk\") pod \"533efba5-8b6c-4726-b0eb-2f7b53ac5ea3\" (UID: \"533efba5-8b6c-4726-b0eb-2f7b53ac5ea3\") "
Oct 30 22:47:19.033631 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:19.033390    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/533efba5-8b6c-4726-b0eb-2f7b53ac5ea3-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/533efba5-8b6c-4726-b0eb-2f7b53ac5ea3-maas-api-tls\") pod \"533efba5-8b6c-4726-b0eb-2f7b53ac5ea3\" (UID: \"533efba5-8b6c-4726-b0eb-2f7b53ac5ea3\") "
Oct 30 22:47:19.048840 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:19.046962    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/533efba5-8b6c-4726-b0eb-2f7b53ac5ea3-openshift-service-ca" pod "533efba5-8b6c-4726-b0eb-2f7b53ac5ea3" (UID: "533efba5-8b6c-4726-b0eb-2f7b53ac5ea3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:47:19.050683 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:19.050434    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/533efba5-8b6c-4726-b0eb-2f7b53ac5ea3-maas-api-tls" pod "533efba5-8b6c-4726-b0eb-2f7b53ac5ea3" (UID: "533efba5-8b6c-4726-b0eb-2f7b53ac5ea3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:47:19.050683 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:19.050510    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/533efba5-8b6c-4726-b0eb-2f7b53ac5ea3-metrics-certs" pod "533efba5-8b6c-4726-b0eb-2f7b53ac5ea3" (UID: "533efba5-8b6c-4726-b0eb-2f7b53ac5ea3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:47:19.054210 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:19.053996    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/533efba5-8b6c-4726-b0eb-2f7b53ac5ea3-kube-api-access-2xsrk" pod "533efba5-8b6c-4726-b0eb-2f7b53ac5ea3" (UID: "533efba5-8b6c-4726-b0eb-2f7b53ac5ea3"). InnerVolumeSpecName "kube-api-access-2xsrk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:47:19.135732 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:19.135693    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/533efba5-8b6c-4726-b0eb-2f7b53ac5ea3-maas-api-tls\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:47:19.135732 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:19.135728    4871 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/533efba5-8b6c-4726-b0eb-2f7b53ac5ea3-metrics-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:47:19.136011 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:19.135744    4871 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/533efba5-8b6c-4726-b0eb-2f7b53ac5ea3-openshift-service-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:47:19.136011 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:19.135757    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2xsrk\" (UniqueName: \"kubernetes.io/projected/533efba5-8b6c-4726-b0eb-2f7b53ac5ea3-kube-api-access-2xsrk\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:47:19.749188 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:19.749149    4871 generic.go:356] "Generic (PLEG): container finished" podID="533efba5-8b6c-4726-b0eb-2f7b53ac5ea3" containerID="6c4e45948ea85a178300e7028108424dfe1a30bcf241ee98312a2892071edaca" exitCode=0
Oct 30 22:47:19.749708 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:19.749205    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2"
Oct 30 22:47:19.749708 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:19.749199    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2" event={"ID":"533efba5-8b6c-4726-b0eb-2f7b53ac5ea3","Type":"ContainerDied","Data":"6c4e45948ea85a178300e7028108424dfe1a30bcf241ee98312a2892071edaca"}
Oct 30 22:47:19.749708 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:19.749321    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2" event={"ID":"533efba5-8b6c-4726-b0eb-2f7b53ac5ea3","Type":"ContainerDied","Data":"8d0175cbed161274e712ae437ef78d6613f6d832faa9eae61113c2444508ce3c"}
Oct 30 22:47:19.749708 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:19.749374    4871 scope.go:122] "RemoveContainer" containerID="6c4e45948ea85a178300e7028108424dfe1a30bcf241ee98312a2892071edaca"
Oct 30 22:47:19.773783 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:19.773756    4871 scope.go:122] "RemoveContainer" containerID="6c4e45948ea85a178300e7028108424dfe1a30bcf241ee98312a2892071edaca"
Oct 30 22:47:19.774163 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:47:19.774134    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c4e45948ea85a178300e7028108424dfe1a30bcf241ee98312a2892071edaca\": container with ID starting with 6c4e45948ea85a178300e7028108424dfe1a30bcf241ee98312a2892071edaca not found: ID does not exist" containerID="6c4e45948ea85a178300e7028108424dfe1a30bcf241ee98312a2892071edaca"
Oct 30 22:47:19.774275 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:19.774163    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c4e45948ea85a178300e7028108424dfe1a30bcf241ee98312a2892071edaca"} err="failed to get container status \"6c4e45948ea85a178300e7028108424dfe1a30bcf241ee98312a2892071edaca\": rpc error: code = NotFound desc = could not find container \"6c4e45948ea85a178300e7028108424dfe1a30bcf241ee98312a2892071edaca\": container with ID starting with 6c4e45948ea85a178300e7028108424dfe1a30bcf241ee98312a2892071edaca not found: ID does not exist"
Oct 30 22:47:19.787977 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:47:19.787937    4871 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod533efba5_8b6c_4726_b0eb_2f7b53ac5ea3.slice/crio-8d0175cbed161274e712ae437ef78d6613f6d832faa9eae61113c2444508ce3c\": RecentStats: unable to find data in memory cache]"
Oct 30 22:47:19.924440 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:19.924394    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2"]
Oct 30 22:47:19.928597 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:19.928569    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ef00a4-6f555bd759-6kmp2"]
Oct 30 22:47:19.961559 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:19.961519    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="533efba5-8b6c-4726-b0eb-2f7b53ac5ea3" path="/var/lib/kubelet/pods/533efba5-8b6c-4726-b0eb-2f7b53ac5ea3/volumes"
Oct 30 22:47:20.398602 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:20.398559    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:20.576695 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:20.576634    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:20.728920 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:20.728829    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:21.334090 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:21.334054    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:24.943126 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:24.943072    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:25.577556 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:25.577517    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:27.942883 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:27.942830    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:28.095482 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:28.095434    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:28.244320 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:28.244178    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"]
Oct 30 22:47:28.244516 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:28.244480    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp" podUID="6cbecab8-bfad-40bf-850a-905e4d365d29" containerName="istio-proxy" containerID="cri-o://915303882d9b027ce1faa0670f764ff80410c5a7e646ad736e9896675b7c9318" gracePeriod=30
Oct 30 22:47:28.253971 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:28.253941    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:30.527040 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:30.527007    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:31.036949 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:31.036915    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:31.197231 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:31.197194    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:31.637534 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:31.637497    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:31.829024 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:31.828988    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:33.567848 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:33.567819    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"
Oct 30 22:47:33.614393 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:33.614356    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6cbecab8-bfad-40bf-850a-905e4d365d29-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cbecab8-bfad-40bf-850a-905e4d365d29-istiod-ca-cert\") pod \"6cbecab8-bfad-40bf-850a-905e4d365d29\" (UID: \"6cbecab8-bfad-40bf-850a-905e4d365d29\") "
Oct 30 22:47:33.614635 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:33.614446    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6cbecab8-bfad-40bf-850a-905e4d365d29-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cbecab8-bfad-40bf-850a-905e4d365d29-istio-podinfo\") pod \"6cbecab8-bfad-40bf-850a-905e4d365d29\" (UID: \"6cbecab8-bfad-40bf-850a-905e4d365d29\") "
Oct 30 22:47:33.614635 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:33.614494    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6cbecab8-bfad-40bf-850a-905e4d365d29-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cbecab8-bfad-40bf-850a-905e4d365d29-istio-envoy\") pod \"6cbecab8-bfad-40bf-850a-905e4d365d29\" (UID: \"6cbecab8-bfad-40bf-850a-905e4d365d29\") "
Oct 30 22:47:33.614635 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:33.614516    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6cbecab8-bfad-40bf-850a-905e4d365d29-istio-token\" (UniqueName: \"kubernetes.io/projected/6cbecab8-bfad-40bf-850a-905e4d365d29-istio-token\") pod \"6cbecab8-bfad-40bf-850a-905e4d365d29\" (UID: \"6cbecab8-bfad-40bf-850a-905e4d365d29\") "
Oct 30 22:47:33.614635 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:33.614576    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6cbecab8-bfad-40bf-850a-905e4d365d29-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cbecab8-bfad-40bf-850a-905e4d365d29-istio-data\") pod \"6cbecab8-bfad-40bf-850a-905e4d365d29\" (UID: \"6cbecab8-bfad-40bf-850a-905e4d365d29\") "
Oct 30 22:47:33.614868 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:33.614663    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6cbecab8-bfad-40bf-850a-905e4d365d29-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cbecab8-bfad-40bf-850a-905e4d365d29-credential-socket\") pod \"6cbecab8-bfad-40bf-850a-905e4d365d29\" (UID: \"6cbecab8-bfad-40bf-850a-905e4d365d29\") "
Oct 30 22:47:33.614868 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:33.614728    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6cbecab8-bfad-40bf-850a-905e4d365d29-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cbecab8-bfad-40bf-850a-905e4d365d29-workload-certs\") pod \"6cbecab8-bfad-40bf-850a-905e4d365d29\" (UID: \"6cbecab8-bfad-40bf-850a-905e4d365d29\") "
Oct 30 22:47:33.614868 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:33.614778    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6cbecab8-bfad-40bf-850a-905e4d365d29-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cbecab8-bfad-40bf-850a-905e4d365d29-workload-socket\") pod \"6cbecab8-bfad-40bf-850a-905e4d365d29\" (UID: \"6cbecab8-bfad-40bf-850a-905e4d365d29\") "
Oct 30 22:47:33.614868 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:33.614828    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6cbecab8-bfad-40bf-850a-905e4d365d29-kube-api-access-xnq9r\" (UniqueName: \"kubernetes.io/projected/6cbecab8-bfad-40bf-850a-905e4d365d29-kube-api-access-xnq9r\") pod \"6cbecab8-bfad-40bf-850a-905e4d365d29\" (UID: \"6cbecab8-bfad-40bf-850a-905e4d365d29\") "
Oct 30 22:47:33.615089 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:33.614935    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cbecab8-bfad-40bf-850a-905e4d365d29-istio-data" pod "6cbecab8-bfad-40bf-850a-905e4d365d29" (UID: "6cbecab8-bfad-40bf-850a-905e4d365d29"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:47:33.615089 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:33.614834    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6cbecab8-bfad-40bf-850a-905e4d365d29-istiod-ca-cert" pod "6cbecab8-bfad-40bf-850a-905e4d365d29" (UID: "6cbecab8-bfad-40bf-850a-905e4d365d29"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:47:33.615208 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:33.615078    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cbecab8-bfad-40bf-850a-905e4d365d29-workload-certs" pod "6cbecab8-bfad-40bf-850a-905e4d365d29" (UID: "6cbecab8-bfad-40bf-850a-905e4d365d29"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:47:33.615208 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:33.615149    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cbecab8-bfad-40bf-850a-905e4d365d29-credential-socket" pod "6cbecab8-bfad-40bf-850a-905e4d365d29" (UID: "6cbecab8-bfad-40bf-850a-905e4d365d29"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:47:33.615208 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:33.615172    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cbecab8-bfad-40bf-850a-905e4d365d29-workload-socket" pod "6cbecab8-bfad-40bf-850a-905e4d365d29" (UID: "6cbecab8-bfad-40bf-850a-905e4d365d29"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:47:33.615814 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:33.615790    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cbecab8-bfad-40bf-850a-905e4d365d29-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:47:33.615911 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:33.615819    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cbecab8-bfad-40bf-850a-905e4d365d29-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:47:33.615911 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:33.615836    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cbecab8-bfad-40bf-850a-905e4d365d29-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:47:33.615911 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:33.615852    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cbecab8-bfad-40bf-850a-905e4d365d29-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:47:33.615911 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:33.615866    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cbecab8-bfad-40bf-850a-905e4d365d29-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:47:33.617047 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:33.617018    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cbecab8-bfad-40bf-850a-905e4d365d29-istio-envoy" pod "6cbecab8-bfad-40bf-850a-905e4d365d29" (UID: "6cbecab8-bfad-40bf-850a-905e4d365d29"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:47:33.617184 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:33.617103    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cbecab8-bfad-40bf-850a-905e4d365d29-istio-token" pod "6cbecab8-bfad-40bf-850a-905e4d365d29" (UID: "6cbecab8-bfad-40bf-850a-905e4d365d29"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:47:33.617330 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:33.617198    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6cbecab8-bfad-40bf-850a-905e4d365d29-istio-podinfo" pod "6cbecab8-bfad-40bf-850a-905e4d365d29" (UID: "6cbecab8-bfad-40bf-850a-905e4d365d29"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:47:33.617396 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:33.617362    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cbecab8-bfad-40bf-850a-905e4d365d29-kube-api-access-xnq9r" pod "6cbecab8-bfad-40bf-850a-905e4d365d29" (UID: "6cbecab8-bfad-40bf-850a-905e4d365d29"). InnerVolumeSpecName "kube-api-access-xnq9r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:47:33.717172 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:33.717130    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cbecab8-bfad-40bf-850a-905e4d365d29-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:47:33.717172 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:33.717162    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cbecab8-bfad-40bf-850a-905e4d365d29-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:47:33.717172 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:33.717172    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6cbecab8-bfad-40bf-850a-905e4d365d29-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:47:33.717172 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:33.717183    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnq9r\" (UniqueName: \"kubernetes.io/projected/6cbecab8-bfad-40bf-850a-905e4d365d29-kube-api-access-xnq9r\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:47:33.875053 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:33.875016    4871 generic.go:356] "Generic (PLEG): container finished" podID="6cbecab8-bfad-40bf-850a-905e4d365d29" containerID="915303882d9b027ce1faa0670f764ff80410c5a7e646ad736e9896675b7c9318" exitCode=0
Oct 30 22:47:33.875261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:33.875078    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"
Oct 30 22:47:33.875261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:33.875104    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp" event={"ID":"6cbecab8-bfad-40bf-850a-905e4d365d29","Type":"ContainerDied","Data":"915303882d9b027ce1faa0670f764ff80410c5a7e646ad736e9896675b7c9318"}
Oct 30 22:47:33.875261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:33.875166    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp" event={"ID":"6cbecab8-bfad-40bf-850a-905e4d365d29","Type":"ContainerDied","Data":"bfb38a5492650097a945fd954d80ea1f607d3456127b840e837c5534d0e5041b"}
Oct 30 22:47:33.875261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:33.875190    4871 scope.go:122] "RemoveContainer" containerID="915303882d9b027ce1faa0670f764ff80410c5a7e646ad736e9896675b7c9318"
Oct 30 22:47:33.887079 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:33.887049    4871 scope.go:122] "RemoveContainer" containerID="a2cfedda456012711321024d33b23567686adf80d7cc9f2cbc8f9f485fbc4d08"
Oct 30 22:47:33.897449 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:33.897428    4871 scope.go:122] "RemoveContainer" containerID="915303882d9b027ce1faa0670f764ff80410c5a7e646ad736e9896675b7c9318"
Oct 30 22:47:33.897781 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:47:33.897758    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"915303882d9b027ce1faa0670f764ff80410c5a7e646ad736e9896675b7c9318\": container with ID starting with 915303882d9b027ce1faa0670f764ff80410c5a7e646ad736e9896675b7c9318 not found: ID does not exist" containerID="915303882d9b027ce1faa0670f764ff80410c5a7e646ad736e9896675b7c9318"
Oct 30 22:47:33.897838 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:33.897792    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"915303882d9b027ce1faa0670f764ff80410c5a7e646ad736e9896675b7c9318"} err="failed to get container status \"915303882d9b027ce1faa0670f764ff80410c5a7e646ad736e9896675b7c9318\": rpc error: code = NotFound desc = could not find container \"915303882d9b027ce1faa0670f764ff80410c5a7e646ad736e9896675b7c9318\": container with ID starting with 915303882d9b027ce1faa0670f764ff80410c5a7e646ad736e9896675b7c9318 not found: ID does not exist"
Oct 30 22:47:33.897838 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:33.897811    4871 scope.go:122] "RemoveContainer" containerID="a2cfedda456012711321024d33b23567686adf80d7cc9f2cbc8f9f485fbc4d08"
Oct 30 22:47:33.898083 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:47:33.898065    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2cfedda456012711321024d33b23567686adf80d7cc9f2cbc8f9f485fbc4d08\": container with ID starting with a2cfedda456012711321024d33b23567686adf80d7cc9f2cbc8f9f485fbc4d08 not found: ID does not exist" containerID="a2cfedda456012711321024d33b23567686adf80d7cc9f2cbc8f9f485fbc4d08"
Oct 30 22:47:33.898134 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:33.898089    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2cfedda456012711321024d33b23567686adf80d7cc9f2cbc8f9f485fbc4d08"} err="failed to get container status \"a2cfedda456012711321024d33b23567686adf80d7cc9f2cbc8f9f485fbc4d08\": rpc error: code = NotFound desc = could not find container \"a2cfedda456012711321024d33b23567686adf80d7cc9f2cbc8f9f485fbc4d08\": container with ID starting with a2cfedda456012711321024d33b23567686adf80d7cc9f2cbc8f9f485fbc4d08 not found: ID does not exist"
Oct 30 22:47:34.050037 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:34.049997    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"]
Oct 30 22:47:34.056194 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:34.056157    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-ef00a4-openshift-default-8d489689c-fgkgp"]
Oct 30 22:47:34.978312 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:34.978280    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:35.569102 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:47:35.569063    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:47:35.961582 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:35.961548    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6cbecab8-bfad-40bf-850a-905e4d365d29" path="/var/lib/kubelet/pods/6cbecab8-bfad-40bf-850a-905e4d365d29/volumes"
Oct 30 22:47:36.474658 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:36.474619    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:36.664388 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:36.664352    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:36.819190 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:36.819093    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:37.429777 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:37.429740    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:38.841315 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:38.841280    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx_8d401654-8a4c-4a79-b2d3-612765312421/extract/0.log"
Oct 30 22:47:38.850550 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:38.850525    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx_8d401654-8a4c-4a79-b2d3-612765312421/extract/0.log"
Oct 30 22:47:38.854846 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:38.854826    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/2.log"
Oct 30 22:47:38.862297 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:38.862277    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzntd_2360fddc-52a5-42a5-82ca-a05aff84d217/ovn-acl-logging/0.log"
Oct 30 22:47:38.863322 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:38.863305    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/2.log"
Oct 30 22:47:38.870833 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:38.870815    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzntd_2360fddc-52a5-42a5-82ca-a05aff84d217/ovn-acl-logging/0.log"
Oct 30 22:47:40.963093 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:40.963052    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:41.116574 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:41.116521    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:41.705694 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:41.705654    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:41.993754 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:41.993625    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:43.593626 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:43.593587    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:43.761036 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:43.760998    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:43.939793 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:43.939748    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:43.944826 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:43.944797    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"]
Oct 30 22:47:43.945072 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:43.945032    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd" podUID="4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e" containerName="istio-proxy" containerID="cri-o://c7a9239fdb8d99e476eda85a3137ed07f23598c4dac44df92daa04806df1045e" gracePeriod=30
Oct 30 22:47:46.345766 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:46.345724    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:48.549302 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:48.549259    4871 patch_prober.go:29] interesting pod/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:47:48.549706 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:48.549323    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd" podUID="4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:47:49.229924 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:49.229898    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"
Oct 30 22:47:49.232520 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:49.232471    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:49.347333 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:49.347293    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-workload-certs\") pod \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\" (UID: \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\") "
Oct 30 22:47:49.347629 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:49.347603    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-istio-podinfo\") pod \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\" (UID: \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\") "
Oct 30 22:47:49.347819 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:49.347803    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-credential-socket\") pod \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\" (UID: \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\") "
Oct 30 22:47:49.348053 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:49.348008    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-istio-data\") pod \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\" (UID: \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\") "
Oct 30 22:47:49.348256 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:49.348211    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-kube-api-access-ghxjb\" (UniqueName: \"kubernetes.io/projected/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-kube-api-access-ghxjb\") pod \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\" (UID: \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\") "
Oct 30 22:47:49.348490 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:49.348470    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-istio-token\" (UniqueName: \"kubernetes.io/projected/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-istio-token\") pod \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\" (UID: \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\") "
Oct 30 22:47:49.348643 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:49.348628    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-istiod-ca-cert\") pod \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\" (UID: \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\") "
Oct 30 22:47:49.348877 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:49.348859    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-workload-socket\") pod \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\" (UID: \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\") "
Oct 30 22:47:49.349431 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:49.349059    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-istiod-ca-cert" pod "4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e" (UID: "4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:47:49.349431 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:49.349155    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-workload-socket" pod "4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e" (UID: "4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:47:49.349431 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:49.349186    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-istio-envoy\") pod \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\" (UID: \"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e\") "
Oct 30 22:47:49.349431 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:49.349230    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-istio-data" pod "4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e" (UID: "4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:47:49.349431 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:49.349299    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-credential-socket" pod "4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e" (UID: "4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:47:49.349431 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:49.349389    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-workload-certs" pod "4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e" (UID: "4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:47:49.350038 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:49.350006    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:47:49.350038 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:49.350032    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:47:49.350186 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:49.350048    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:47:49.350186 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:49.350064    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:47:49.350186 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:49.350078    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:47:49.352681 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:49.352610    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-istio-podinfo" pod "4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e" (UID: "4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:47:49.352681 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:49.352641    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-istio-token" pod "4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e" (UID: "4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:47:49.353538 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:49.353493    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-kube-api-access-ghxjb" pod "4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e" (UID: "4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e"). InnerVolumeSpecName "kube-api-access-ghxjb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:47:49.354625 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:49.354597    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-istio-envoy" pod "4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e" (UID: "4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:47:49.387118 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:49.387075    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:49.451959 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:49.451921    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghxjb\" (UniqueName: \"kubernetes.io/projected/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-kube-api-access-ghxjb\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:47:49.451959 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:49.451958    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:47:49.451959 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:49.451967    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:47:49.452357 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:49.451976    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:47:50.030049 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:50.030012    4871 generic.go:356] "Generic (PLEG): container finished" podID="4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e" containerID="c7a9239fdb8d99e476eda85a3137ed07f23598c4dac44df92daa04806df1045e" exitCode=0
Oct 30 22:47:50.030586 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:50.030065    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"
Oct 30 22:47:50.030586 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:50.030094    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd" event={"ID":"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e","Type":"ContainerDied","Data":"c7a9239fdb8d99e476eda85a3137ed07f23598c4dac44df92daa04806df1045e"}
Oct 30 22:47:50.030586 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:50.030232    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd" event={"ID":"4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e","Type":"ContainerDied","Data":"b69af78e14c8e1a382cfb7c2d5ed9fc0d71fec3b15862d711c0b45bee9adf687"}
Oct 30 22:47:50.030586 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:50.030297    4871 scope.go:122] "RemoveContainer" containerID="c7a9239fdb8d99e476eda85a3137ed07f23598c4dac44df92daa04806df1045e"
Oct 30 22:47:50.044100 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:50.044072    4871 scope.go:122] "RemoveContainer" containerID="c7a9239fdb8d99e476eda85a3137ed07f23598c4dac44df92daa04806df1045e"
Oct 30 22:47:50.044481 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:47:50.044457    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7a9239fdb8d99e476eda85a3137ed07f23598c4dac44df92daa04806df1045e\": container with ID starting with c7a9239fdb8d99e476eda85a3137ed07f23598c4dac44df92daa04806df1045e not found: ID does not exist" containerID="c7a9239fdb8d99e476eda85a3137ed07f23598c4dac44df92daa04806df1045e"
Oct 30 22:47:50.044563 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:50.044490    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7a9239fdb8d99e476eda85a3137ed07f23598c4dac44df92daa04806df1045e"} err="failed to get container status \"c7a9239fdb8d99e476eda85a3137ed07f23598c4dac44df92daa04806df1045e\": rpc error: code = NotFound desc = could not find container \"c7a9239fdb8d99e476eda85a3137ed07f23598c4dac44df92daa04806df1045e\": container with ID starting with c7a9239fdb8d99e476eda85a3137ed07f23598c4dac44df92daa04806df1045e not found: ID does not exist"
Oct 30 22:47:50.257013 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:50.256928    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"]
Oct 30 22:47:50.307734 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:50.307697    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-6e0b99-openshift-default-849bf65785-pgxxd"]
Oct 30 22:47:51.361789 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:51.361752    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:51.961874 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:51.961834    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e" path="/var/lib/kubelet/pods/4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e/volumes"
Oct 30 22:47:53.002610 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:53.002569    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:53.933274 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:53.933225    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:54.449302 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:54.449265    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:54.623305 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:54.620964    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:54.787048 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:54.786917    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:56.223520 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:56.223458    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:58.275827 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:58.275747    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:58.433249 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:58.433196    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:58.669060 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:47:58.669021    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:00.466230 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:00.466192    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x"]
Oct 30 22:48:00.466792 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:00.466417    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x" podUID="2f8e8b67-0b00-4fb5-b125-865f980b97c0" containerName="maas-api" containerID="cri-o://9ecd4a9578c236b550f6d07365c4905661d1018ba839e6effad0eb9bb43bf35c" gracePeriod=30
Oct 30 22:48:00.567947 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:00.567908    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:00.801575 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:00.801532    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x"
Oct 30 22:48:00.831083 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:00.831043    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2f8e8b67-0b00-4fb5-b125-865f980b97c0-kube-api-access-pc9d9\" (UniqueName: \"kubernetes.io/projected/2f8e8b67-0b00-4fb5-b125-865f980b97c0-kube-api-access-pc9d9\") pod \"2f8e8b67-0b00-4fb5-b125-865f980b97c0\" (UID: \"2f8e8b67-0b00-4fb5-b125-865f980b97c0\") "
Oct 30 22:48:00.831280 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:00.831152    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2f8e8b67-0b00-4fb5-b125-865f980b97c0-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f8e8b67-0b00-4fb5-b125-865f980b97c0-maas-api-tls\") pod \"2f8e8b67-0b00-4fb5-b125-865f980b97c0\" (UID: \"2f8e8b67-0b00-4fb5-b125-865f980b97c0\") "
Oct 30 22:48:00.831280 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:00.831228    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2f8e8b67-0b00-4fb5-b125-865f980b97c0-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f8e8b67-0b00-4fb5-b125-865f980b97c0-metrics-certs\") pod \"2f8e8b67-0b00-4fb5-b125-865f980b97c0\" (UID: \"2f8e8b67-0b00-4fb5-b125-865f980b97c0\") "
Oct 30 22:48:00.831442 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:00.831419    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2f8e8b67-0b00-4fb5-b125-865f980b97c0-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f8e8b67-0b00-4fb5-b125-865f980b97c0-openshift-service-ca\") pod \"2f8e8b67-0b00-4fb5-b125-865f980b97c0\" (UID: \"2f8e8b67-0b00-4fb5-b125-865f980b97c0\") "
Oct 30 22:48:00.831881 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:00.831852    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f8e8b67-0b00-4fb5-b125-865f980b97c0-openshift-service-ca" pod "2f8e8b67-0b00-4fb5-b125-865f980b97c0" (UID: "2f8e8b67-0b00-4fb5-b125-865f980b97c0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:48:00.832027 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:00.832009    4871 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f8e8b67-0b00-4fb5-b125-865f980b97c0-openshift-service-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:48:00.833389 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:00.833362    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f8e8b67-0b00-4fb5-b125-865f980b97c0-metrics-certs" pod "2f8e8b67-0b00-4fb5-b125-865f980b97c0" (UID: "2f8e8b67-0b00-4fb5-b125-865f980b97c0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:48:00.833498 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:00.833427    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f8e8b67-0b00-4fb5-b125-865f980b97c0-kube-api-access-pc9d9" pod "2f8e8b67-0b00-4fb5-b125-865f980b97c0" (UID: "2f8e8b67-0b00-4fb5-b125-865f980b97c0"). InnerVolumeSpecName "kube-api-access-pc9d9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:48:00.833498 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:00.833473    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f8e8b67-0b00-4fb5-b125-865f980b97c0-maas-api-tls" pod "2f8e8b67-0b00-4fb5-b125-865f980b97c0" (UID: "2f8e8b67-0b00-4fb5-b125-865f980b97c0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:48:00.933968 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:00.933932    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pc9d9\" (UniqueName: \"kubernetes.io/projected/2f8e8b67-0b00-4fb5-b125-865f980b97c0-kube-api-access-pc9d9\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:48:00.937353 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:00.937320    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f8e8b67-0b00-4fb5-b125-865f980b97c0-maas-api-tls\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:48:00.937353 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:00.937356    4871 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f8e8b67-0b00-4fb5-b125-865f980b97c0-metrics-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:48:01.119813 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:01.119774    4871 generic.go:356] "Generic (PLEG): container finished" podID="2f8e8b67-0b00-4fb5-b125-865f980b97c0" containerID="9ecd4a9578c236b550f6d07365c4905661d1018ba839e6effad0eb9bb43bf35c" exitCode=0
Oct 30 22:48:01.120011 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:01.119821    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x" event={"ID":"2f8e8b67-0b00-4fb5-b125-865f980b97c0","Type":"ContainerDied","Data":"9ecd4a9578c236b550f6d07365c4905661d1018ba839e6effad0eb9bb43bf35c"}
Oct 30 22:48:01.120011 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:01.119835    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x"
Oct 30 22:48:01.120011 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:01.119884    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x" event={"ID":"2f8e8b67-0b00-4fb5-b125-865f980b97c0","Type":"ContainerDied","Data":"df280d0c38b87422b5865fa07aaed94b3f1f031a9efd0bdc6b539922ace0ba5f"}
Oct 30 22:48:01.120011 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:01.119911    4871 scope.go:122] "RemoveContainer" containerID="9ecd4a9578c236b550f6d07365c4905661d1018ba839e6effad0eb9bb43bf35c"
Oct 30 22:48:01.131672 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:01.131639    4871 scope.go:122] "RemoveContainer" containerID="9ecd4a9578c236b550f6d07365c4905661d1018ba839e6effad0eb9bb43bf35c"
Oct 30 22:48:01.132073 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:48:01.132050    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ecd4a9578c236b550f6d07365c4905661d1018ba839e6effad0eb9bb43bf35c\": container with ID starting with 9ecd4a9578c236b550f6d07365c4905661d1018ba839e6effad0eb9bb43bf35c not found: ID does not exist" containerID="9ecd4a9578c236b550f6d07365c4905661d1018ba839e6effad0eb9bb43bf35c"
Oct 30 22:48:01.132159 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:01.132085    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ecd4a9578c236b550f6d07365c4905661d1018ba839e6effad0eb9bb43bf35c"} err="failed to get container status \"9ecd4a9578c236b550f6d07365c4905661d1018ba839e6effad0eb9bb43bf35c\": rpc error: code = NotFound desc = could not find container \"9ecd4a9578c236b550f6d07365c4905661d1018ba839e6effad0eb9bb43bf35c\": container with ID starting with 9ecd4a9578c236b550f6d07365c4905661d1018ba839e6effad0eb9bb43bf35c not found: ID does not exist"
Oct 30 22:48:01.301137 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:01.301097    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x"]
Oct 30 22:48:01.306602 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:01.306573    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-323082-889d48fb-9pd9x"]
Oct 30 22:48:01.961623 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:01.961588    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2f8e8b67-0b00-4fb5-b125-865f980b97c0" path="/var/lib/kubelet/pods/2f8e8b67-0b00-4fb5-b125-865f980b97c0/volumes"
Oct 30 22:48:02.199300 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:02.199262    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:02.363291 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:02.363257    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:02.574124 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:02.574089    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:03.135389 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:03.135357    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:04.430033 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:04.429992    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:04.614996 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:04.614960    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:05.571983 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:48:05.571937    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:48:07.346960 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:07.346922    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:09.396729 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:09.396692    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:09.557254 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:09.557175    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:09.767116 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:09.767024    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"]
Oct 30 22:48:09.767314 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:09.767270    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r" podUID="b873ef79-e292-4496-a9ec-8d2ec09f65f6" containerName="istio-proxy" containerID="cri-o://09228f534a4ff4ea72dbbc34d77b8564be9aa0a1545a816d176e013d80dbac23" gracePeriod=30
Oct 30 22:48:09.780698 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:09.780664    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:10.225698 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:10.225667    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:12.246923 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:12.246882    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:15.052710 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.052633    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"
Oct 30 22:48:15.173216 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.173182    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b873ef79-e292-4496-a9ec-8d2ec09f65f6-istio-token\" (UniqueName: \"kubernetes.io/projected/b873ef79-e292-4496-a9ec-8d2ec09f65f6-istio-token\") pod \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\" (UID: \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\") "
Oct 30 22:48:15.173414 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.173287    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b873ef79-e292-4496-a9ec-8d2ec09f65f6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b873ef79-e292-4496-a9ec-8d2ec09f65f6-workload-socket\") pod \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\" (UID: \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\") "
Oct 30 22:48:15.173414 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.173333    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b873ef79-e292-4496-a9ec-8d2ec09f65f6-kube-api-access-xxt85\" (UniqueName: \"kubernetes.io/projected/b873ef79-e292-4496-a9ec-8d2ec09f65f6-kube-api-access-xxt85\") pod \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\" (UID: \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\") "
Oct 30 22:48:15.173414 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.173378    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b873ef79-e292-4496-a9ec-8d2ec09f65f6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b873ef79-e292-4496-a9ec-8d2ec09f65f6-workload-certs\") pod \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\" (UID: \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\") "
Oct 30 22:48:15.173589 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.173443    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b873ef79-e292-4496-a9ec-8d2ec09f65f6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b873ef79-e292-4496-a9ec-8d2ec09f65f6-istio-podinfo\") pod \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\" (UID: \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\") "
Oct 30 22:48:15.173589 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.173518    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b873ef79-e292-4496-a9ec-8d2ec09f65f6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b873ef79-e292-4496-a9ec-8d2ec09f65f6-istio-data\") pod \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\" (UID: \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\") "
Oct 30 22:48:15.173589 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.173553    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b873ef79-e292-4496-a9ec-8d2ec09f65f6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b873ef79-e292-4496-a9ec-8d2ec09f65f6-credential-socket\") pod \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\" (UID: \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\") "
Oct 30 22:48:15.173765 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.173669    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b873ef79-e292-4496-a9ec-8d2ec09f65f6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b873ef79-e292-4496-a9ec-8d2ec09f65f6-istio-envoy\") pod \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\" (UID: \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\") "
Oct 30 22:48:15.173823 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.173752    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b873ef79-e292-4496-a9ec-8d2ec09f65f6-workload-socket" pod "b873ef79-e292-4496-a9ec-8d2ec09f65f6" (UID: "b873ef79-e292-4496-a9ec-8d2ec09f65f6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:48:15.173823 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.173790    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b873ef79-e292-4496-a9ec-8d2ec09f65f6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b873ef79-e292-4496-a9ec-8d2ec09f65f6-istiod-ca-cert\") pod \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\" (UID: \"b873ef79-e292-4496-a9ec-8d2ec09f65f6\") "
Oct 30 22:48:15.173939 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.173843    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b873ef79-e292-4496-a9ec-8d2ec09f65f6-workload-certs" pod "b873ef79-e292-4496-a9ec-8d2ec09f65f6" (UID: "b873ef79-e292-4496-a9ec-8d2ec09f65f6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:48:15.173939 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.173899    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b873ef79-e292-4496-a9ec-8d2ec09f65f6-istio-data" pod "b873ef79-e292-4496-a9ec-8d2ec09f65f6" (UID: "b873ef79-e292-4496-a9ec-8d2ec09f65f6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:48:15.173939 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.173929    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b873ef79-e292-4496-a9ec-8d2ec09f65f6-credential-socket" pod "b873ef79-e292-4496-a9ec-8d2ec09f65f6" (UID: "b873ef79-e292-4496-a9ec-8d2ec09f65f6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:48:15.174261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.174213    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b873ef79-e292-4496-a9ec-8d2ec09f65f6-istiod-ca-cert" pod "b873ef79-e292-4496-a9ec-8d2ec09f65f6" (UID: "b873ef79-e292-4496-a9ec-8d2ec09f65f6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:48:15.174381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.174362    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b873ef79-e292-4496-a9ec-8d2ec09f65f6-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:48:15.174453 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.174386    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b873ef79-e292-4496-a9ec-8d2ec09f65f6-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:48:15.174453 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.174402    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b873ef79-e292-4496-a9ec-8d2ec09f65f6-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:48:15.174453 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.174416    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b873ef79-e292-4496-a9ec-8d2ec09f65f6-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:48:15.174453 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.174429    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b873ef79-e292-4496-a9ec-8d2ec09f65f6-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:48:15.175603 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.175580    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b873ef79-e292-4496-a9ec-8d2ec09f65f6-istio-token" pod "b873ef79-e292-4496-a9ec-8d2ec09f65f6" (UID: "b873ef79-e292-4496-a9ec-8d2ec09f65f6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:48:15.176029 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.176005    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b873ef79-e292-4496-a9ec-8d2ec09f65f6-istio-podinfo" pod "b873ef79-e292-4496-a9ec-8d2ec09f65f6" (UID: "b873ef79-e292-4496-a9ec-8d2ec09f65f6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:48:15.176133 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.176118    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b873ef79-e292-4496-a9ec-8d2ec09f65f6-istio-envoy" pod "b873ef79-e292-4496-a9ec-8d2ec09f65f6" (UID: "b873ef79-e292-4496-a9ec-8d2ec09f65f6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:48:15.176591 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.176559    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b873ef79-e292-4496-a9ec-8d2ec09f65f6-kube-api-access-xxt85" pod "b873ef79-e292-4496-a9ec-8d2ec09f65f6" (UID: "b873ef79-e292-4496-a9ec-8d2ec09f65f6"). InnerVolumeSpecName "kube-api-access-xxt85". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:48:15.204543 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.204507    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8"]
Oct 30 22:48:15.272168 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.272127    4871 generic.go:356] "Generic (PLEG): container finished" podID="b873ef79-e292-4496-a9ec-8d2ec09f65f6" containerID="09228f534a4ff4ea72dbbc34d77b8564be9aa0a1545a816d176e013d80dbac23" exitCode=0
Oct 30 22:48:15.272394 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.272251    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r" event={"ID":"b873ef79-e292-4496-a9ec-8d2ec09f65f6","Type":"ContainerDied","Data":"09228f534a4ff4ea72dbbc34d77b8564be9aa0a1545a816d176e013d80dbac23"}
Oct 30 22:48:15.272394 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.272315    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r" event={"ID":"b873ef79-e292-4496-a9ec-8d2ec09f65f6","Type":"ContainerDied","Data":"787047c924b6ea60881427ecc59565dd1c9380ab7585f2918ce7977afb310cad"}
Oct 30 22:48:15.272394 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.272337    4871 scope.go:122] "RemoveContainer" containerID="09228f534a4ff4ea72dbbc34d77b8564be9aa0a1545a816d176e013d80dbac23"
Oct 30 22:48:15.272584 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.272452    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"
Oct 30 22:48:15.275482 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.275449    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b873ef79-e292-4496-a9ec-8d2ec09f65f6-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:48:15.275482 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.275487    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b873ef79-e292-4496-a9ec-8d2ec09f65f6-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:48:15.275729 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.275503    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b873ef79-e292-4496-a9ec-8d2ec09f65f6-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:48:15.275729 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.275519    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xxt85\" (UniqueName: \"kubernetes.io/projected/b873ef79-e292-4496-a9ec-8d2ec09f65f6-kube-api-access-xxt85\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:48:15.285551 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.285524    4871 scope.go:122] "RemoveContainer" containerID="23e5724be2c58f004dd3c8be76117e3f3fa9d18f096a485d1a48c673db63e0f6"
Oct 30 22:48:15.297037 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.297015    4871 scope.go:122] "RemoveContainer" containerID="09228f534a4ff4ea72dbbc34d77b8564be9aa0a1545a816d176e013d80dbac23"
Oct 30 22:48:15.297140 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.297037    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:15.297383 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:48:15.297363    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09228f534a4ff4ea72dbbc34d77b8564be9aa0a1545a816d176e013d80dbac23\": container with ID starting with 09228f534a4ff4ea72dbbc34d77b8564be9aa0a1545a816d176e013d80dbac23 not found: ID does not exist" containerID="09228f534a4ff4ea72dbbc34d77b8564be9aa0a1545a816d176e013d80dbac23"
Oct 30 22:48:15.297481 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.297396    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09228f534a4ff4ea72dbbc34d77b8564be9aa0a1545a816d176e013d80dbac23"} err="failed to get container status \"09228f534a4ff4ea72dbbc34d77b8564be9aa0a1545a816d176e013d80dbac23\": rpc error: code = NotFound desc = could not find container \"09228f534a4ff4ea72dbbc34d77b8564be9aa0a1545a816d176e013d80dbac23\": container with ID starting with 09228f534a4ff4ea72dbbc34d77b8564be9aa0a1545a816d176e013d80dbac23 not found: ID does not exist"
Oct 30 22:48:15.297481 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.297421    4871 scope.go:122] "RemoveContainer" containerID="23e5724be2c58f004dd3c8be76117e3f3fa9d18f096a485d1a48c673db63e0f6"
Oct 30 22:48:15.297714 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:48:15.297695    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23e5724be2c58f004dd3c8be76117e3f3fa9d18f096a485d1a48c673db63e0f6\": container with ID starting with 23e5724be2c58f004dd3c8be76117e3f3fa9d18f096a485d1a48c673db63e0f6 not found: ID does not exist" containerID="23e5724be2c58f004dd3c8be76117e3f3fa9d18f096a485d1a48c673db63e0f6"
Oct 30 22:48:15.297797 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.297721    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23e5724be2c58f004dd3c8be76117e3f3fa9d18f096a485d1a48c673db63e0f6"} err="failed to get container status \"23e5724be2c58f004dd3c8be76117e3f3fa9d18f096a485d1a48c673db63e0f6\": rpc error: code = NotFound desc = could not find container \"23e5724be2c58f004dd3c8be76117e3f3fa9d18f096a485d1a48c673db63e0f6\": container with ID starting with 23e5724be2c58f004dd3c8be76117e3f3fa9d18f096a485d1a48c673db63e0f6 not found: ID does not exist"
Oct 30 22:48:15.496993 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.496951    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"]
Oct 30 22:48:15.500738 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.500704    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-323082-openshift-default-78f46c58c5-nq76r"]
Oct 30 22:48:15.961956 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:15.961911    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b873ef79-e292-4496-a9ec-8d2ec09f65f6" path="/var/lib/kubelet/pods/b873ef79-e292-4496-a9ec-8d2ec09f65f6/volumes"
Oct 30 22:48:17.155611 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:17.155576    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:22.136752 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:22.136712    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:22.394333 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:22.394230    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-3c4802-5f6bddd989-bl95r"]
Oct 30 22:48:22.509462 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:22.509421    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-3c4802-7bd9747b5f-sfrmg"]
Oct 30 22:48:22.771302 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:22.771170    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:22.958500 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:22.958458    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:23.586338 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:23.586300    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:24.561277 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:24.561233    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:26.136232 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:26.136198    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:27.394750 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:27.394701    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w1-3c4802-5f6bddd989-bl95r" podUID="7c957c3f-753d-4d57-b724-19fdce296860" containerName="payload-processing" containerID="cri-o://92cf645164171857e2a832f2e90c13f838d580ed26ec8667a285d715ab83c116" gracePeriod=25
Oct 30 22:48:27.511429 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:27.511378    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-3c4802-7bd9747b5f-sfrmg" podUID="7ea0c6d1-5765-43c8-95c1-19473cc909e2" containerName="payload-pre-processing" containerID="cri-o://99505ec2c53198b22b5f80dd37e60ec33f320237aeaa0fc4ec9208a3794fcd91" gracePeriod=25
Oct 30 22:48:27.608540 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:27.608502    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:27.914702 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:27.914651    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:28.069146 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:28.069112    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:28.564925 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:28.564886    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:30.226453 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:30.226405    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8" podUID="84329f90-dc35-41dc-b593-12c556e80f73" containerName="main" containerID="cri-o://ca1e9c929acaca04d466dfd6fbcc43cf1562705a2b1366924bdc4e16027eda85" gracePeriod=45
Oct 30 22:48:30.398731 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:30.398698    4871 generic.go:356] "Generic (PLEG): container finished" podID="84329f90-dc35-41dc-b593-12c556e80f73" containerID="ca1e9c929acaca04d466dfd6fbcc43cf1562705a2b1366924bdc4e16027eda85" exitCode=0
Oct 30 22:48:30.398947 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:30.398768    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8" event={"ID":"84329f90-dc35-41dc-b593-12c556e80f73","Type":"ContainerDied","Data":"ca1e9c929acaca04d466dfd6fbcc43cf1562705a2b1366924bdc4e16027eda85"}
Oct 30 22:48:30.532583 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:30.532541    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8"
Oct 30 22:48:30.618833 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:30.618786    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/84329f90-dc35-41dc-b593-12c556e80f73-tls-certs\" (UniqueName: \"kubernetes.io/secret/84329f90-dc35-41dc-b593-12c556e80f73-tls-certs\") pod \"84329f90-dc35-41dc-b593-12c556e80f73\" (UID: \"84329f90-dc35-41dc-b593-12c556e80f73\") "
Oct 30 22:48:30.619053 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:30.618864    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/84329f90-dc35-41dc-b593-12c556e80f73-kube-api-access-2vxg7\" (UniqueName: \"kubernetes.io/projected/84329f90-dc35-41dc-b593-12c556e80f73-kube-api-access-2vxg7\") pod \"84329f90-dc35-41dc-b593-12c556e80f73\" (UID: \"84329f90-dc35-41dc-b593-12c556e80f73\") "
Oct 30 22:48:30.619053 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:30.618959    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84329f90-dc35-41dc-b593-12c556e80f73-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84329f90-dc35-41dc-b593-12c556e80f73-model-cache\") pod \"84329f90-dc35-41dc-b593-12c556e80f73\" (UID: \"84329f90-dc35-41dc-b593-12c556e80f73\") "
Oct 30 22:48:30.619053 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:30.619021    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84329f90-dc35-41dc-b593-12c556e80f73-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84329f90-dc35-41dc-b593-12c556e80f73-tmp-dir\") pod \"84329f90-dc35-41dc-b593-12c556e80f73\" (UID: \"84329f90-dc35-41dc-b593-12c556e80f73\") "
Oct 30 22:48:30.619225 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:30.619103    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84329f90-dc35-41dc-b593-12c556e80f73-home\" (UniqueName: \"kubernetes.io/empty-dir/84329f90-dc35-41dc-b593-12c556e80f73-home\") pod \"84329f90-dc35-41dc-b593-12c556e80f73\" (UID: \"84329f90-dc35-41dc-b593-12c556e80f73\") "
Oct 30 22:48:30.619225 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:30.619146    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84329f90-dc35-41dc-b593-12c556e80f73-dshm\" (UniqueName: \"kubernetes.io/empty-dir/84329f90-dc35-41dc-b593-12c556e80f73-dshm\") pod \"84329f90-dc35-41dc-b593-12c556e80f73\" (UID: \"84329f90-dc35-41dc-b593-12c556e80f73\") "
Oct 30 22:48:30.621695 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:30.621593    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84329f90-dc35-41dc-b593-12c556e80f73-model-cache" pod "84329f90-dc35-41dc-b593-12c556e80f73" (UID: "84329f90-dc35-41dc-b593-12c556e80f73"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:48:30.622005 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:30.621948    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84329f90-dc35-41dc-b593-12c556e80f73-tmp-dir" pod "84329f90-dc35-41dc-b593-12c556e80f73" (UID: "84329f90-dc35-41dc-b593-12c556e80f73"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:48:30.622805 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:30.622783    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84329f90-dc35-41dc-b593-12c556e80f73-home" pod "84329f90-dc35-41dc-b593-12c556e80f73" (UID: "84329f90-dc35-41dc-b593-12c556e80f73"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:48:30.624299 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:30.624275    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84329f90-dc35-41dc-b593-12c556e80f73-dshm" pod "84329f90-dc35-41dc-b593-12c556e80f73" (UID: "84329f90-dc35-41dc-b593-12c556e80f73"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:48:30.629586 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:30.629561    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84329f90-dc35-41dc-b593-12c556e80f73-tls-certs" pod "84329f90-dc35-41dc-b593-12c556e80f73" (UID: "84329f90-dc35-41dc-b593-12c556e80f73"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:48:30.629756 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:30.629730    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84329f90-dc35-41dc-b593-12c556e80f73-kube-api-access-2vxg7" pod "84329f90-dc35-41dc-b593-12c556e80f73" (UID: "84329f90-dc35-41dc-b593-12c556e80f73"). InnerVolumeSpecName "kube-api-access-2vxg7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:48:30.721501 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:30.721452    4871 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84329f90-dc35-41dc-b593-12c556e80f73-tls-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:48:30.721501 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:30.721502    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vxg7\" (UniqueName: \"kubernetes.io/projected/84329f90-dc35-41dc-b593-12c556e80f73-kube-api-access-2vxg7\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:48:30.721737 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:30.721518    4871 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84329f90-dc35-41dc-b593-12c556e80f73-model-cache\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:48:30.721737 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:30.721533    4871 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84329f90-dc35-41dc-b593-12c556e80f73-tmp-dir\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:48:30.721737 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:30.721547    4871 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84329f90-dc35-41dc-b593-12c556e80f73-home\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:48:30.721737 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:30.721559    4871 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84329f90-dc35-41dc-b593-12c556e80f73-dshm\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:48:31.408292 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:31.408259    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8"
Oct 30 22:48:31.408292 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:31.408273    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8" event={"ID":"84329f90-dc35-41dc-b593-12c556e80f73","Type":"ContainerDied","Data":"a05af2779fb51d291e15834dbe63876fa4c5384c1e340a620970f057943f1a4b"}
Oct 30 22:48:31.408873 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:31.408339    4871 scope.go:122] "RemoveContainer" containerID="ca1e9c929acaca04d466dfd6fbcc43cf1562705a2b1366924bdc4e16027eda85"
Oct 30 22:48:31.537119 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:31.537074    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-3c4802-7bd9747b5f-sfrmg" podUID="7ea0c6d1-5765-43c8-95c1-19473cc909e2" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:48:31.537340 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:31.537084    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w1-3c4802-5f6bddd989-bl95r" podUID="7c957c3f-753d-4d57-b724-19fdce296860" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:48:31.617810 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:31.617770    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8"]
Oct 30 22:48:31.621733 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:31.621705    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-3c4802/premium-w1-3c4802-kserve-7c5849d557-fs6r8"]
Oct 30 22:48:31.965908 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:31.965851    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="84329f90-dc35-41dc-b593-12c556e80f73" path="/var/lib/kubelet/pods/84329f90-dc35-41dc-b593-12c556e80f73/volumes"
Oct 30 22:48:32.039102 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:32.039056    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:33.529141 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:33.529106    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:33.819077 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:33.818981    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:34.017009 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:34.016976    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:34.465049 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:34.465006    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:34.974267 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:34.974212    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:35.163511 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:35.163477    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:35.331343 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:35.331188    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:35.570735 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:48:35.570699    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:48:39.035037 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:39.034961    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:40.854333 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:40.854290    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:41.044834 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:41.044802    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:41.246014 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:41.245959    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:41.536582 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:41.536491    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w1-3c4802-5f6bddd989-bl95r" podUID="7c957c3f-753d-4d57-b724-19fdce296860" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:48:41.536781 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:41.536510    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-3c4802-7bd9747b5f-sfrmg" podUID="7ea0c6d1-5765-43c8-95c1-19473cc909e2" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:48:43.954915 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:43.954882    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:47.594095 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:47.594062    4871 generic.go:356] "Generic (PLEG): container finished" podID="7c957c3f-753d-4d57-b724-19fdce296860" containerID="92cf645164171857e2a832f2e90c13f838d580ed26ec8667a285d715ab83c116" exitCode=0
Oct 30 22:48:47.594598 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:47.594403    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-3c4802-5f6bddd989-bl95r" event={"ID":"7c957c3f-753d-4d57-b724-19fdce296860","Type":"ContainerDied","Data":"92cf645164171857e2a832f2e90c13f838d580ed26ec8667a285d715ab83c116"}
Oct 30 22:48:47.685390 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:47.685361    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-3c4802-5f6bddd989-bl95r"
Oct 30 22:48:47.776418 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:47.776392    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-3c4802-7bd9747b5f-sfrmg"
Oct 30 22:48:47.814766 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:47.814727    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7c957c3f-753d-4d57-b724-19fdce296860-kube-api-access-wb5sp\" (UniqueName: \"kubernetes.io/projected/7c957c3f-753d-4d57-b724-19fdce296860-kube-api-access-wb5sp\") pod \"7c957c3f-753d-4d57-b724-19fdce296860\" (UID: \"7c957c3f-753d-4d57-b724-19fdce296860\") "
Oct 30 22:48:47.814956 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:47.814853    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7c957c3f-753d-4d57-b724-19fdce296860-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c957c3f-753d-4d57-b724-19fdce296860-plugins-config-volume\") pod \"7c957c3f-753d-4d57-b724-19fdce296860\" (UID: \"7c957c3f-753d-4d57-b724-19fdce296860\") "
Oct 30 22:48:47.816089 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:47.816060    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c957c3f-753d-4d57-b724-19fdce296860-plugins-config-volume" pod "7c957c3f-753d-4d57-b724-19fdce296860" (UID: "7c957c3f-753d-4d57-b724-19fdce296860"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:48:47.818448 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:47.818413    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c957c3f-753d-4d57-b724-19fdce296860-kube-api-access-wb5sp" pod "7c957c3f-753d-4d57-b724-19fdce296860" (UID: "7c957c3f-753d-4d57-b724-19fdce296860"). InnerVolumeSpecName "kube-api-access-wb5sp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:48:47.916286 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:47.916180    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7ea0c6d1-5765-43c8-95c1-19473cc909e2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ea0c6d1-5765-43c8-95c1-19473cc909e2-plugins-config-volume\") pod \"7ea0c6d1-5765-43c8-95c1-19473cc909e2\" (UID: \"7ea0c6d1-5765-43c8-95c1-19473cc909e2\") "
Oct 30 22:48:47.916477 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:47.916450    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7ea0c6d1-5765-43c8-95c1-19473cc909e2-kube-api-access-fzm5z\" (UniqueName: \"kubernetes.io/projected/7ea0c6d1-5765-43c8-95c1-19473cc909e2-kube-api-access-fzm5z\") pod \"7ea0c6d1-5765-43c8-95c1-19473cc909e2\" (UID: \"7ea0c6d1-5765-43c8-95c1-19473cc909e2\") "
Oct 30 22:48:47.916658 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:47.916627    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ea0c6d1-5765-43c8-95c1-19473cc909e2-plugins-config-volume" pod "7ea0c6d1-5765-43c8-95c1-19473cc909e2" (UID: "7ea0c6d1-5765-43c8-95c1-19473cc909e2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:48:47.916898 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:47.916880    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wb5sp\" (UniqueName: \"kubernetes.io/projected/7c957c3f-753d-4d57-b724-19fdce296860-kube-api-access-wb5sp\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:48:47.916968 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:47.916906    4871 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c957c3f-753d-4d57-b724-19fdce296860-plugins-config-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:48:47.916968 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:47.916924    4871 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ea0c6d1-5765-43c8-95c1-19473cc909e2-plugins-config-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:48:47.919186 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:47.919148    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ea0c6d1-5765-43c8-95c1-19473cc909e2-kube-api-access-fzm5z" pod "7ea0c6d1-5765-43c8-95c1-19473cc909e2" (UID: "7ea0c6d1-5765-43c8-95c1-19473cc909e2"). InnerVolumeSpecName "kube-api-access-fzm5z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:48:48.018640 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:48.018608    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fzm5z\" (UniqueName: \"kubernetes.io/projected/7ea0c6d1-5765-43c8-95c1-19473cc909e2-kube-api-access-fzm5z\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:48:48.076585 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:48.076544    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:48.226373 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:48.226266    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:48.603357 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:48.603274    4871 generic.go:356] "Generic (PLEG): container finished" podID="7ea0c6d1-5765-43c8-95c1-19473cc909e2" containerID="99505ec2c53198b22b5f80dd37e60ec33f320237aeaa0fc4ec9208a3794fcd91" exitCode=0
Oct 30 22:48:48.603357 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:48.603310    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-3c4802-7bd9747b5f-sfrmg"
Oct 30 22:48:48.603357 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:48.603330    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-3c4802-7bd9747b5f-sfrmg" event={"ID":"7ea0c6d1-5765-43c8-95c1-19473cc909e2","Type":"ContainerDied","Data":"99505ec2c53198b22b5f80dd37e60ec33f320237aeaa0fc4ec9208a3794fcd91"}
Oct 30 22:48:48.603919 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:48.603384    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-3c4802-7bd9747b5f-sfrmg" event={"ID":"7ea0c6d1-5765-43c8-95c1-19473cc909e2","Type":"ContainerDied","Data":"da4997784b87c22f69d56cafa749ffeadae17ed24d7cefd677b0fdec83eccf29"}
Oct 30 22:48:48.603919 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:48.603406    4871 scope.go:122] "RemoveContainer" containerID="99505ec2c53198b22b5f80dd37e60ec33f320237aeaa0fc4ec9208a3794fcd91"
Oct 30 22:48:48.605430 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:48.605406    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-3c4802-5f6bddd989-bl95r" event={"ID":"7c957c3f-753d-4d57-b724-19fdce296860","Type":"ContainerDied","Data":"dc9d0168fb2f679391f8297a048ad0af3f0f02d63ea3d02647083aaf00cd63cb"}
Oct 30 22:48:48.605573 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:48.605554    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-3c4802-5f6bddd989-bl95r"
Oct 30 22:48:48.621351 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:48.621329    4871 scope.go:122] "RemoveContainer" containerID="99505ec2c53198b22b5f80dd37e60ec33f320237aeaa0fc4ec9208a3794fcd91"
Oct 30 22:48:48.621679 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:48:48.621660    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99505ec2c53198b22b5f80dd37e60ec33f320237aeaa0fc4ec9208a3794fcd91\": container with ID starting with 99505ec2c53198b22b5f80dd37e60ec33f320237aeaa0fc4ec9208a3794fcd91 not found: ID does not exist" containerID="99505ec2c53198b22b5f80dd37e60ec33f320237aeaa0fc4ec9208a3794fcd91"
Oct 30 22:48:48.621785 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:48.621689    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99505ec2c53198b22b5f80dd37e60ec33f320237aeaa0fc4ec9208a3794fcd91"} err="failed to get container status \"99505ec2c53198b22b5f80dd37e60ec33f320237aeaa0fc4ec9208a3794fcd91\": rpc error: code = NotFound desc = could not find container \"99505ec2c53198b22b5f80dd37e60ec33f320237aeaa0fc4ec9208a3794fcd91\": container with ID starting with 99505ec2c53198b22b5f80dd37e60ec33f320237aeaa0fc4ec9208a3794fcd91 not found: ID does not exist"
Oct 30 22:48:48.621785 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:48.621710    4871 scope.go:122] "RemoveContainer" containerID="92cf645164171857e2a832f2e90c13f838d580ed26ec8667a285d715ab83c116"
Oct 30 22:48:48.777511 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:48.777477    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-3c4802-7bd9747b5f-sfrmg"]
Oct 30 22:48:48.832270 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:48.832221    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-3c4802-7bd9747b5f-sfrmg"]
Oct 30 22:48:48.842948 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:48.842912    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:48.923970 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:48.922728    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-3c4802-5f6bddd989-bl95r"]
Oct 30 22:48:48.928822 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:48.928791    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-3c4802-5f6bddd989-bl95r"]
Oct 30 22:48:49.961013 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:49.960980    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7c957c3f-753d-4d57-b724-19fdce296860" path="/var/lib/kubelet/pods/7c957c3f-753d-4d57-b724-19fdce296860/volumes"
Oct 30 22:48:49.961463 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:49.961448    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7ea0c6d1-5765-43c8-95c1-19473cc909e2" path="/var/lib/kubelet/pods/7ea0c6d1-5765-43c8-95c1-19473cc909e2/volumes"
Oct 30 22:48:52.653261 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:52.652387    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:57.807633 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:57.807592    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:57.963748 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:57.963698    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:58.248646 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:58.248609    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z"]
Oct 30 22:48:58.248900 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:58.248877    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z" podUID="9b7e498c-c51e-4f65-8f1c-e1fb615f278c" containerName="maas-api" containerID="cri-o://0cda477b231f4d2be9ef394e19dcce21503e1a4961852a3bfb86feeb329889e5" gracePeriod=30
Oct 30 22:48:58.375762 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:58.375640    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:58.590965 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:58.590936    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z"
Oct 30 22:48:58.698916 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:58.698873    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9b7e498c-c51e-4f65-8f1c-e1fb615f278c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b7e498c-c51e-4f65-8f1c-e1fb615f278c-openshift-service-ca\") pod \"9b7e498c-c51e-4f65-8f1c-e1fb615f278c\" (UID: \"9b7e498c-c51e-4f65-8f1c-e1fb615f278c\") "
Oct 30 22:48:58.699804 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:58.699777    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9b7e498c-c51e-4f65-8f1c-e1fb615f278c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b7e498c-c51e-4f65-8f1c-e1fb615f278c-metrics-certs\") pod \"9b7e498c-c51e-4f65-8f1c-e1fb615f278c\" (UID: \"9b7e498c-c51e-4f65-8f1c-e1fb615f278c\") "
Oct 30 22:48:58.700022 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:58.700002    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9b7e498c-c51e-4f65-8f1c-e1fb615f278c-kube-api-access-cgv7q\" (UniqueName: \"kubernetes.io/projected/9b7e498c-c51e-4f65-8f1c-e1fb615f278c-kube-api-access-cgv7q\") pod \"9b7e498c-c51e-4f65-8f1c-e1fb615f278c\" (UID: \"9b7e498c-c51e-4f65-8f1c-e1fb615f278c\") "
Oct 30 22:48:58.700168 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:58.700153    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9b7e498c-c51e-4f65-8f1c-e1fb615f278c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b7e498c-c51e-4f65-8f1c-e1fb615f278c-maas-api-tls\") pod \"9b7e498c-c51e-4f65-8f1c-e1fb615f278c\" (UID: \"9b7e498c-c51e-4f65-8f1c-e1fb615f278c\") "
Oct 30 22:48:58.702732 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:58.699592    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b7e498c-c51e-4f65-8f1c-e1fb615f278c-openshift-service-ca" pod "9b7e498c-c51e-4f65-8f1c-e1fb615f278c" (UID: "9b7e498c-c51e-4f65-8f1c-e1fb615f278c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:48:58.740433 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:58.740387    4871 generic.go:356] "Generic (PLEG): container finished" podID="9b7e498c-c51e-4f65-8f1c-e1fb615f278c" containerID="0cda477b231f4d2be9ef394e19dcce21503e1a4961852a3bfb86feeb329889e5" exitCode=0
Oct 30 22:48:58.740730 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:58.740697    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z" event={"ID":"9b7e498c-c51e-4f65-8f1c-e1fb615f278c","Type":"ContainerDied","Data":"0cda477b231f4d2be9ef394e19dcce21503e1a4961852a3bfb86feeb329889e5"}
Oct 30 22:48:58.740853 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:58.740812    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z" event={"ID":"9b7e498c-c51e-4f65-8f1c-e1fb615f278c","Type":"ContainerDied","Data":"b66f046681fc056a9e0a10a63639724122b5f47d146c139470865fa13d8192aa"}
Oct 30 22:48:58.740944 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:58.740858    4871 scope.go:122] "RemoveContainer" containerID="0cda477b231f4d2be9ef394e19dcce21503e1a4961852a3bfb86feeb329889e5"
Oct 30 22:48:58.741086 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:58.741069    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z"
Oct 30 22:48:58.767489 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:58.761526    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b7e498c-c51e-4f65-8f1c-e1fb615f278c-maas-api-tls" pod "9b7e498c-c51e-4f65-8f1c-e1fb615f278c" (UID: "9b7e498c-c51e-4f65-8f1c-e1fb615f278c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:48:58.767489 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:58.763075    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b7e498c-c51e-4f65-8f1c-e1fb615f278c-metrics-certs" pod "9b7e498c-c51e-4f65-8f1c-e1fb615f278c" (UID: "9b7e498c-c51e-4f65-8f1c-e1fb615f278c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:48:58.767489 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:58.766170    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b7e498c-c51e-4f65-8f1c-e1fb615f278c-kube-api-access-cgv7q" pod "9b7e498c-c51e-4f65-8f1c-e1fb615f278c" (UID: "9b7e498c-c51e-4f65-8f1c-e1fb615f278c"). InnerVolumeSpecName "kube-api-access-cgv7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:48:58.779795 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:58.779758    4871 scope.go:122] "RemoveContainer" containerID="0cda477b231f4d2be9ef394e19dcce21503e1a4961852a3bfb86feeb329889e5"
Oct 30 22:48:58.780203 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:48:58.780167    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cda477b231f4d2be9ef394e19dcce21503e1a4961852a3bfb86feeb329889e5\": container with ID starting with 0cda477b231f4d2be9ef394e19dcce21503e1a4961852a3bfb86feeb329889e5 not found: ID does not exist" containerID="0cda477b231f4d2be9ef394e19dcce21503e1a4961852a3bfb86feeb329889e5"
Oct 30 22:48:58.780727 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:58.780216    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cda477b231f4d2be9ef394e19dcce21503e1a4961852a3bfb86feeb329889e5"} err="failed to get container status \"0cda477b231f4d2be9ef394e19dcce21503e1a4961852a3bfb86feeb329889e5\": rpc error: code = NotFound desc = could not find container \"0cda477b231f4d2be9ef394e19dcce21503e1a4961852a3bfb86feeb329889e5\": container with ID starting with 0cda477b231f4d2be9ef394e19dcce21503e1a4961852a3bfb86feeb329889e5 not found: ID does not exist"
Oct 30 22:48:58.801946 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:58.801912    4871 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b7e498c-c51e-4f65-8f1c-e1fb615f278c-openshift-service-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:48:58.801946 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:58.801946    4871 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b7e498c-c51e-4f65-8f1c-e1fb615f278c-metrics-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:48:58.802149 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:58.801956    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cgv7q\" (UniqueName: \"kubernetes.io/projected/9b7e498c-c51e-4f65-8f1c-e1fb615f278c-kube-api-access-cgv7q\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:48:58.802149 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:58.801967    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b7e498c-c51e-4f65-8f1c-e1fb615f278c-maas-api-tls\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:48:59.171122 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:59.171084    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z"]
Oct 30 22:48:59.246448 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:59.246397    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-1a5f4a-66dbd9cf7d-x824z"]
Oct 30 22:48:59.790876 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:59.790809    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:59.957523 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:59.957451    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:59.961895 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:48:59.961863    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9b7e498c-c51e-4f65-8f1c-e1fb615f278c" path="/var/lib/kubelet/pods/9b7e498c-c51e-4f65-8f1c-e1fb615f278c/volumes"
Oct 30 22:49:00.219652 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:00.219618    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:00.371230 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:00.371194    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:00.875306 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:00.875270    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:05.032056 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:05.032023    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:05.568990 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:49:05.568953    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:49:07.217432 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:07.217400    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:07.400590 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:07.400559    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:07.602772 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:07.602683    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:09.889872 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:09.889832    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:14.020731 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:14.020695    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:14.676031 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:14.675997    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:14.830782 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:14.830738    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:15.055341 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:15.055230    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:19.683764 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:19.683730    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:22.749594 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:22.749559    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:22.933265 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:22.923969    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:24.235031 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:24.234995    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:28.765824 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:28.765767    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:33.420600 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:33.420557    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:35.569522 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:49:35.569478    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:49:37.883758 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:37.883717    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:42.516739 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:42.516698    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:46.583481 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:46.583444    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:47.672764 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:47.672724    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:48.030491 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:48.030398    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:48.344295 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:48.344168    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:48.801486 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:48.801438    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:49.091365 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:49.091196    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:49.255859 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:49.255814    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:49.493966 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:49.493930    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:50.174542 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:50.174506    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:50.789553 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:50.789510    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:51.097102 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:51.097010    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:51.433030 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:51.432985    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:52.004099 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:52.004067    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:53.777960 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:53.777920    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:55.212393 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:55.212353    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:55.381302 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:55.376635    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:55.592757 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:55.592663    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:55.747440 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:55.747394    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:56.767553 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:56.767519    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:56.940954 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:56.940912    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:57.112459 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:57.112427    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"]
Oct 30 22:49:57.112685 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:57.112660    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm" podUID="f3798606-a072-44a5-891c-4588de3c6ff1" containerName="istio-proxy" containerID="cri-o://e89e1ca44b46da72cb1105b3c1750a576e4432ad389c7f2122844b9f9c5568dc" gracePeriod=30
Oct 30 22:49:57.127592 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:57.127560    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:58.781856 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:58.781814    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:58.941088 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:58.941052    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:59.150303 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:59.150273    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:59.620059 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:49:59.620026    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:01.060552 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:01.060521    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:02.357374 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:02.357336    4871 generic.go:356] "Generic (PLEG): container finished" podID="f3798606-a072-44a5-891c-4588de3c6ff1" containerID="e89e1ca44b46da72cb1105b3c1750a576e4432ad389c7f2122844b9f9c5568dc" exitCode=0
Oct 30 22:50:02.357888 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:02.357497    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm" event={"ID":"f3798606-a072-44a5-891c-4588de3c6ff1","Type":"ContainerDied","Data":"e89e1ca44b46da72cb1105b3c1750a576e4432ad389c7f2122844b9f9c5568dc"}
Oct 30 22:50:02.357888 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:02.357559    4871 scope.go:122] "RemoveContainer" containerID="32f8069ac79b7a359568aa080a026b48d32973596ce134cef15097cb136fcddf"
Oct 30 22:50:02.389678 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:02.389657    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"
Oct 30 22:50:02.522215 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:02.522165    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f3798606-a072-44a5-891c-4588de3c6ff1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3798606-a072-44a5-891c-4588de3c6ff1-workload-certs\") pod \"f3798606-a072-44a5-891c-4588de3c6ff1\" (UID: \"f3798606-a072-44a5-891c-4588de3c6ff1\") "
Oct 30 22:50:02.522428 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:02.522265    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f3798606-a072-44a5-891c-4588de3c6ff1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3798606-a072-44a5-891c-4588de3c6ff1-istio-envoy\") pod \"f3798606-a072-44a5-891c-4588de3c6ff1\" (UID: \"f3798606-a072-44a5-891c-4588de3c6ff1\") "
Oct 30 22:50:02.522428 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:02.522356    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f3798606-a072-44a5-891c-4588de3c6ff1-istio-token\" (UniqueName: \"kubernetes.io/projected/f3798606-a072-44a5-891c-4588de3c6ff1-istio-token\") pod \"f3798606-a072-44a5-891c-4588de3c6ff1\" (UID: \"f3798606-a072-44a5-891c-4588de3c6ff1\") "
Oct 30 22:50:02.522428 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:02.522397    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f3798606-a072-44a5-891c-4588de3c6ff1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3798606-a072-44a5-891c-4588de3c6ff1-istiod-ca-cert\") pod \"f3798606-a072-44a5-891c-4588de3c6ff1\" (UID: \"f3798606-a072-44a5-891c-4588de3c6ff1\") "
Oct 30 22:50:02.522665 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:02.522475    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f3798606-a072-44a5-891c-4588de3c6ff1-kube-api-access-sxhnc\" (UniqueName: \"kubernetes.io/projected/f3798606-a072-44a5-891c-4588de3c6ff1-kube-api-access-sxhnc\") pod \"f3798606-a072-44a5-891c-4588de3c6ff1\" (UID: \"f3798606-a072-44a5-891c-4588de3c6ff1\") "
Oct 30 22:50:02.522665 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:02.522520    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f3798606-a072-44a5-891c-4588de3c6ff1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3798606-a072-44a5-891c-4588de3c6ff1-workload-socket\") pod \"f3798606-a072-44a5-891c-4588de3c6ff1\" (UID: \"f3798606-a072-44a5-891c-4588de3c6ff1\") "
Oct 30 22:50:02.522665 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:02.522556    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3798606-a072-44a5-891c-4588de3c6ff1-workload-certs" pod "f3798606-a072-44a5-891c-4588de3c6ff1" (UID: "f3798606-a072-44a5-891c-4588de3c6ff1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:02.522665 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:02.522610    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f3798606-a072-44a5-891c-4588de3c6ff1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3798606-a072-44a5-891c-4588de3c6ff1-credential-socket\") pod \"f3798606-a072-44a5-891c-4588de3c6ff1\" (UID: \"f3798606-a072-44a5-891c-4588de3c6ff1\") "
Oct 30 22:50:02.522665 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:02.522652    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f3798606-a072-44a5-891c-4588de3c6ff1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3798606-a072-44a5-891c-4588de3c6ff1-istio-podinfo\") pod \"f3798606-a072-44a5-891c-4588de3c6ff1\" (UID: \"f3798606-a072-44a5-891c-4588de3c6ff1\") "
Oct 30 22:50:02.522957 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:02.522696    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f3798606-a072-44a5-891c-4588de3c6ff1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3798606-a072-44a5-891c-4588de3c6ff1-istio-data\") pod \"f3798606-a072-44a5-891c-4588de3c6ff1\" (UID: \"f3798606-a072-44a5-891c-4588de3c6ff1\") "
Oct 30 22:50:02.523141 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:02.523112    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3798606-a072-44a5-891c-4588de3c6ff1-workload-socket" pod "f3798606-a072-44a5-891c-4588de3c6ff1" (UID: "f3798606-a072-44a5-891c-4588de3c6ff1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:02.523297 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:02.523272    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3798606-a072-44a5-891c-4588de3c6ff1-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:02.523469 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:02.523452    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3798606-a072-44a5-891c-4588de3c6ff1-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:02.523639 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:02.523550    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3798606-a072-44a5-891c-4588de3c6ff1-istiod-ca-cert" pod "f3798606-a072-44a5-891c-4588de3c6ff1" (UID: "f3798606-a072-44a5-891c-4588de3c6ff1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:50:02.523639 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:02.523615    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3798606-a072-44a5-891c-4588de3c6ff1-istio-data" pod "f3798606-a072-44a5-891c-4588de3c6ff1" (UID: "f3798606-a072-44a5-891c-4588de3c6ff1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:02.523809 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:02.523744    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3798606-a072-44a5-891c-4588de3c6ff1-credential-socket" pod "f3798606-a072-44a5-891c-4588de3c6ff1" (UID: "f3798606-a072-44a5-891c-4588de3c6ff1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:02.525073 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:02.524918    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f3798606-a072-44a5-891c-4588de3c6ff1-istio-podinfo" pod "f3798606-a072-44a5-891c-4588de3c6ff1" (UID: "f3798606-a072-44a5-891c-4588de3c6ff1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:50:02.525073 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:02.525078    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3798606-a072-44a5-891c-4588de3c6ff1-istio-token" pod "f3798606-a072-44a5-891c-4588de3c6ff1" (UID: "f3798606-a072-44a5-891c-4588de3c6ff1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:50:02.525287 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:02.525132    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3798606-a072-44a5-891c-4588de3c6ff1-istio-envoy" pod "f3798606-a072-44a5-891c-4588de3c6ff1" (UID: "f3798606-a072-44a5-891c-4588de3c6ff1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:02.525548 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:02.525529    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3798606-a072-44a5-891c-4588de3c6ff1-kube-api-access-sxhnc" pod "f3798606-a072-44a5-891c-4588de3c6ff1" (UID: "f3798606-a072-44a5-891c-4588de3c6ff1"). InnerVolumeSpecName "kube-api-access-sxhnc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:50:02.624621 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:02.624577    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3798606-a072-44a5-891c-4588de3c6ff1-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:02.624621 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:02.624617    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3798606-a072-44a5-891c-4588de3c6ff1-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:02.624621 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:02.624629    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3798606-a072-44a5-891c-4588de3c6ff1-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:02.624879 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:02.624638    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3798606-a072-44a5-891c-4588de3c6ff1-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:02.624879 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:02.624647    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3798606-a072-44a5-891c-4588de3c6ff1-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:02.624879 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:02.624655    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3798606-a072-44a5-891c-4588de3c6ff1-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:02.624879 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:02.624664    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxhnc\" (UniqueName: \"kubernetes.io/projected/f3798606-a072-44a5-891c-4588de3c6ff1-kube-api-access-sxhnc\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:03.365892 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:03.365857    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"
Oct 30 22:50:03.365892 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:03.365871    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm" event={"ID":"f3798606-a072-44a5-891c-4588de3c6ff1","Type":"ContainerDied","Data":"8c5aff5aa9bc6716895b2434407ac7568f5b438616c51bf3dd427e059edf63e6"}
Oct 30 22:50:03.366498 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:03.365929    4871 scope.go:122] "RemoveContainer" containerID="e89e1ca44b46da72cb1105b3c1750a576e4432ad389c7f2122844b9f9c5568dc"
Oct 30 22:50:03.663790 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:03.663701    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"]
Oct 30 22:50:03.669519 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:03.669493    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-3c4802-openshift-default-865cb5887f-spsrm"]
Oct 30 22:50:03.961027 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:03.960950    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f3798606-a072-44a5-891c-4588de3c6ff1" path="/var/lib/kubelet/pods/f3798606-a072-44a5-891c-4588de3c6ff1/volumes"
Oct 30 22:50:04.584192 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:04.584157    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj"]
Oct 30 22:50:04.765996 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:04.765959    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:05.569270 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:50:05.569204    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:50:05.714477 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:05.714443    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:08.360258 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:08.360214    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:08.534573 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:08.533752    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:09.765390 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:09.765353    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:11.280981 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:11.280949    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:11.548966 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:11.548863    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:11.752713 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:11.752684    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:11.755637 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:11.755602    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-26743a-fcb59c9c7-s4rnl"]
Oct 30 22:50:11.918462 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:11.918417    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-26743a-75976bb55f-5vl8h"]
Oct 30 22:50:12.169540 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:12.169424    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:12.328265 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:12.328211    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:16.715406 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:16.715369    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:16.766446 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:16.766364    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-26743a-fcb59c9c7-s4rnl" podUID="ff4cdc4a-dff3-4e33-beae-ce10bb079a80" containerName="payload-processing" containerID="cri-o://5e9e0f59cacf0089232321e2abaae89cf0c298a82dcdb0da7aee0c1b0239b8de" gracePeriod=25
Oct 30 22:50:16.919679 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:16.919614    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-26743a-75976bb55f-5vl8h" podUID="9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c" containerName="payload-pre-processing" containerID="cri-o://ebceb8b74027404ef3569a82d81e803020729f8925c67ec748db13d39979a7ce" gracePeriod=25
Oct 30 22:50:17.324394 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:17.324352    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-26743a-fcb59c9c7-s4rnl" podUID="ff4cdc4a-dff3-4e33-beae-ce10bb079a80" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:50:17.324611 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:17.324356    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-26743a-75976bb55f-5vl8h" podUID="9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:50:18.262567 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:18.262490    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:18.423220 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:18.423185    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:18.626763 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:18.626724    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"]
Oct 30 22:50:18.626956 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:18.626940    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm" podUID="38bc129a-15c0-4019-b32d-e4b62e79b513" containerName="istio-proxy" containerID="cri-o://6fe5f2325e0c3a0a3e1f738b5e4abddc17cc27e8f7dad3e9a579f116b7455739" gracePeriod=30
Oct 30 22:50:18.639233 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:18.639198    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:19.606015 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:19.605969    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj" podUID="93c5282f-81b6-4518-b673-68f527fdfd04" containerName="main" containerID="cri-o://bd7d8b23763e165b81b338e03491a3abe248fc3bcd83d7c4b18849ff02d4396d" gracePeriod=45
Oct 30 22:50:19.644197 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:19.644160    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj" podUID="93c5282f-81b6-4518-b673-68f527fdfd04" containerName="main" probeResult="failure" output="Get \"https://10.133.0.104:8000/ready\": dial tcp 10.133.0.104:8000: connect: connection refused"
Oct 30 22:50:19.891583 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:19.891552    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj"
Oct 30 22:50:19.948260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:19.948200    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93c5282f-81b6-4518-b673-68f527fdfd04-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93c5282f-81b6-4518-b673-68f527fdfd04-tmp-dir\") pod \"93c5282f-81b6-4518-b673-68f527fdfd04\" (UID: \"93c5282f-81b6-4518-b673-68f527fdfd04\") "
Oct 30 22:50:19.948500 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:19.948327    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/93c5282f-81b6-4518-b673-68f527fdfd04-kube-api-access-sjx47\" (UniqueName: \"kubernetes.io/projected/93c5282f-81b6-4518-b673-68f527fdfd04-kube-api-access-sjx47\") pod \"93c5282f-81b6-4518-b673-68f527fdfd04\" (UID: \"93c5282f-81b6-4518-b673-68f527fdfd04\") "
Oct 30 22:50:19.948500 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:19.948411    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93c5282f-81b6-4518-b673-68f527fdfd04-dshm\" (UniqueName: \"kubernetes.io/empty-dir/93c5282f-81b6-4518-b673-68f527fdfd04-dshm\") pod \"93c5282f-81b6-4518-b673-68f527fdfd04\" (UID: \"93c5282f-81b6-4518-b673-68f527fdfd04\") "
Oct 30 22:50:19.948500 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:19.948450    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/93c5282f-81b6-4518-b673-68f527fdfd04-tls-certs\" (UniqueName: \"kubernetes.io/secret/93c5282f-81b6-4518-b673-68f527fdfd04-tls-certs\") pod \"93c5282f-81b6-4518-b673-68f527fdfd04\" (UID: \"93c5282f-81b6-4518-b673-68f527fdfd04\") "
Oct 30 22:50:19.948669 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:19.948563    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93c5282f-81b6-4518-b673-68f527fdfd04-home\" (UniqueName: \"kubernetes.io/empty-dir/93c5282f-81b6-4518-b673-68f527fdfd04-home\") pod \"93c5282f-81b6-4518-b673-68f527fdfd04\" (UID: \"93c5282f-81b6-4518-b673-68f527fdfd04\") "
Oct 30 22:50:19.948669 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:19.948616    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93c5282f-81b6-4518-b673-68f527fdfd04-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93c5282f-81b6-4518-b673-68f527fdfd04-model-cache\") pod \"93c5282f-81b6-4518-b673-68f527fdfd04\" (UID: \"93c5282f-81b6-4518-b673-68f527fdfd04\") "
Oct 30 22:50:19.950051 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:19.950010    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93c5282f-81b6-4518-b673-68f527fdfd04-model-cache" pod "93c5282f-81b6-4518-b673-68f527fdfd04" (UID: "93c5282f-81b6-4518-b673-68f527fdfd04"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:19.950302 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:19.950269    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93c5282f-81b6-4518-b673-68f527fdfd04-tmp-dir" pod "93c5282f-81b6-4518-b673-68f527fdfd04" (UID: "93c5282f-81b6-4518-b673-68f527fdfd04"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:19.951870 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:19.951836    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93c5282f-81b6-4518-b673-68f527fdfd04-home" pod "93c5282f-81b6-4518-b673-68f527fdfd04" (UID: "93c5282f-81b6-4518-b673-68f527fdfd04"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:19.957609 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:19.957567    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93c5282f-81b6-4518-b673-68f527fdfd04-dshm" pod "93c5282f-81b6-4518-b673-68f527fdfd04" (UID: "93c5282f-81b6-4518-b673-68f527fdfd04"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:19.958327 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:19.958300    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93c5282f-81b6-4518-b673-68f527fdfd04-kube-api-access-sjx47" pod "93c5282f-81b6-4518-b673-68f527fdfd04" (UID: "93c5282f-81b6-4518-b673-68f527fdfd04"). InnerVolumeSpecName "kube-api-access-sjx47". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:50:19.958735 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:19.958717    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93c5282f-81b6-4518-b673-68f527fdfd04-tls-certs" pod "93c5282f-81b6-4518-b673-68f527fdfd04" (UID: "93c5282f-81b6-4518-b673-68f527fdfd04"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:50:20.050799 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:20.050760    4871 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93c5282f-81b6-4518-b673-68f527fdfd04-home\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:20.050799 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:20.050796    4871 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93c5282f-81b6-4518-b673-68f527fdfd04-model-cache\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:20.051014 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:20.050828    4871 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93c5282f-81b6-4518-b673-68f527fdfd04-tmp-dir\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:20.051014 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:20.050839    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjx47\" (UniqueName: \"kubernetes.io/projected/93c5282f-81b6-4518-b673-68f527fdfd04-kube-api-access-sjx47\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:20.051014 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:20.050850    4871 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93c5282f-81b6-4518-b673-68f527fdfd04-dshm\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:20.051014 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:20.050858    4871 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93c5282f-81b6-4518-b673-68f527fdfd04-tls-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:20.604531 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:20.604488    4871 generic.go:356] "Generic (PLEG): container finished" podID="93c5282f-81b6-4518-b673-68f527fdfd04" containerID="bd7d8b23763e165b81b338e03491a3abe248fc3bcd83d7c4b18849ff02d4396d" exitCode=0
Oct 30 22:50:20.604738 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:20.604608    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj"
Oct 30 22:50:20.604738 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:20.604630    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj" event={"ID":"93c5282f-81b6-4518-b673-68f527fdfd04","Type":"ContainerDied","Data":"bd7d8b23763e165b81b338e03491a3abe248fc3bcd83d7c4b18849ff02d4396d"}
Oct 30 22:50:20.604738 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:20.604677    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj" event={"ID":"93c5282f-81b6-4518-b673-68f527fdfd04","Type":"ContainerDied","Data":"2e1d917d7df390ba2af095cdd3fa122aa8b7a492cc203940ce9b63cd3cd4c8cf"}
Oct 30 22:50:20.604738 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:20.604696    4871 scope.go:122] "RemoveContainer" containerID="bd7d8b23763e165b81b338e03491a3abe248fc3bcd83d7c4b18849ff02d4396d"
Oct 30 22:50:20.617484 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:20.617444    4871 scope.go:122] "RemoveContainer" containerID="bd7d8b23763e165b81b338e03491a3abe248fc3bcd83d7c4b18849ff02d4396d"
Oct 30 22:50:20.617883 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:50:20.617860    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd7d8b23763e165b81b338e03491a3abe248fc3bcd83d7c4b18849ff02d4396d\": container with ID starting with bd7d8b23763e165b81b338e03491a3abe248fc3bcd83d7c4b18849ff02d4396d not found: ID does not exist" containerID="bd7d8b23763e165b81b338e03491a3abe248fc3bcd83d7c4b18849ff02d4396d"
Oct 30 22:50:20.617947 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:20.617897    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd7d8b23763e165b81b338e03491a3abe248fc3bcd83d7c4b18849ff02d4396d"} err="failed to get container status \"bd7d8b23763e165b81b338e03491a3abe248fc3bcd83d7c4b18849ff02d4396d\": rpc error: code = NotFound desc = could not find container \"bd7d8b23763e165b81b338e03491a3abe248fc3bcd83d7c4b18849ff02d4396d\": container with ID starting with bd7d8b23763e165b81b338e03491a3abe248fc3bcd83d7c4b18849ff02d4396d not found: ID does not exist"
Oct 30 22:50:20.778929 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:20.778892    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj"]
Oct 30 22:50:20.830868 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:20.830831    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-26743a/distinct-w4-26743a-kserve-85985748cd-vl2fj"]
Oct 30 22:50:21.738838 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:21.738805    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:21.909223 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:21.909155    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:21.961652 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:21.961616    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="93c5282f-81b6-4518-b673-68f527fdfd04" path="/var/lib/kubelet/pods/93c5282f-81b6-4518-b673-68f527fdfd04/volumes"
Oct 30 22:50:23.178581 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:23.178523    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:23.805608 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:23.805478    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:24.026087 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.026023    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:24.071045 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.071018    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"
Oct 30 22:50:24.214931 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.214833    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/38bc129a-15c0-4019-b32d-e4b62e79b513-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38bc129a-15c0-4019-b32d-e4b62e79b513-workload-socket\") pod \"38bc129a-15c0-4019-b32d-e4b62e79b513\" (UID: \"38bc129a-15c0-4019-b32d-e4b62e79b513\") "
Oct 30 22:50:24.214931 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.214931    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/38bc129a-15c0-4019-b32d-e4b62e79b513-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38bc129a-15c0-4019-b32d-e4b62e79b513-istio-data\") pod \"38bc129a-15c0-4019-b32d-e4b62e79b513\" (UID: \"38bc129a-15c0-4019-b32d-e4b62e79b513\") "
Oct 30 22:50:24.215551 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.214958    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/38bc129a-15c0-4019-b32d-e4b62e79b513-kube-api-access-qzlf6\" (UniqueName: \"kubernetes.io/projected/38bc129a-15c0-4019-b32d-e4b62e79b513-kube-api-access-qzlf6\") pod \"38bc129a-15c0-4019-b32d-e4b62e79b513\" (UID: \"38bc129a-15c0-4019-b32d-e4b62e79b513\") "
Oct 30 22:50:24.215551 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.214990    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/38bc129a-15c0-4019-b32d-e4b62e79b513-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38bc129a-15c0-4019-b32d-e4b62e79b513-istio-envoy\") pod \"38bc129a-15c0-4019-b32d-e4b62e79b513\" (UID: \"38bc129a-15c0-4019-b32d-e4b62e79b513\") "
Oct 30 22:50:24.215551 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.215036    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/38bc129a-15c0-4019-b32d-e4b62e79b513-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38bc129a-15c0-4019-b32d-e4b62e79b513-istio-podinfo\") pod \"38bc129a-15c0-4019-b32d-e4b62e79b513\" (UID: \"38bc129a-15c0-4019-b32d-e4b62e79b513\") "
Oct 30 22:50:24.215551 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.215056    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/38bc129a-15c0-4019-b32d-e4b62e79b513-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38bc129a-15c0-4019-b32d-e4b62e79b513-workload-certs\") pod \"38bc129a-15c0-4019-b32d-e4b62e79b513\" (UID: \"38bc129a-15c0-4019-b32d-e4b62e79b513\") "
Oct 30 22:50:24.215551 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.215103    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/38bc129a-15c0-4019-b32d-e4b62e79b513-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38bc129a-15c0-4019-b32d-e4b62e79b513-istiod-ca-cert\") pod \"38bc129a-15c0-4019-b32d-e4b62e79b513\" (UID: \"38bc129a-15c0-4019-b32d-e4b62e79b513\") "
Oct 30 22:50:24.215551 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.215145    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/38bc129a-15c0-4019-b32d-e4b62e79b513-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38bc129a-15c0-4019-b32d-e4b62e79b513-credential-socket\") pod \"38bc129a-15c0-4019-b32d-e4b62e79b513\" (UID: \"38bc129a-15c0-4019-b32d-e4b62e79b513\") "
Oct 30 22:50:24.215551 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.215170    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/38bc129a-15c0-4019-b32d-e4b62e79b513-istio-token\" (UniqueName: \"kubernetes.io/projected/38bc129a-15c0-4019-b32d-e4b62e79b513-istio-token\") pod \"38bc129a-15c0-4019-b32d-e4b62e79b513\" (UID: \"38bc129a-15c0-4019-b32d-e4b62e79b513\") "
Oct 30 22:50:24.215551 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.215214    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38bc129a-15c0-4019-b32d-e4b62e79b513-workload-socket" pod "38bc129a-15c0-4019-b32d-e4b62e79b513" (UID: "38bc129a-15c0-4019-b32d-e4b62e79b513"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:24.215551 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.215356    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38bc129a-15c0-4019-b32d-e4b62e79b513-istio-data" pod "38bc129a-15c0-4019-b32d-e4b62e79b513" (UID: "38bc129a-15c0-4019-b32d-e4b62e79b513"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:24.215551 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.215538    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38bc129a-15c0-4019-b32d-e4b62e79b513-credential-socket" pod "38bc129a-15c0-4019-b32d-e4b62e79b513" (UID: "38bc129a-15c0-4019-b32d-e4b62e79b513"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:24.215551 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.215550    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38bc129a-15c0-4019-b32d-e4b62e79b513-workload-certs" pod "38bc129a-15c0-4019-b32d-e4b62e79b513" (UID: "38bc129a-15c0-4019-b32d-e4b62e79b513"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:24.216165 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.215779    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38bc129a-15c0-4019-b32d-e4b62e79b513-istiod-ca-cert" pod "38bc129a-15c0-4019-b32d-e4b62e79b513" (UID: "38bc129a-15c0-4019-b32d-e4b62e79b513"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:50:24.216165 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.216132    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38bc129a-15c0-4019-b32d-e4b62e79b513-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:24.216165 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.216155    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38bc129a-15c0-4019-b32d-e4b62e79b513-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:24.216441 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.216168    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38bc129a-15c0-4019-b32d-e4b62e79b513-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:24.216441 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.216184    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38bc129a-15c0-4019-b32d-e4b62e79b513-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:24.216441 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.216200    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38bc129a-15c0-4019-b32d-e4b62e79b513-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:24.217658 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.217625    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/38bc129a-15c0-4019-b32d-e4b62e79b513-istio-podinfo" pod "38bc129a-15c0-4019-b32d-e4b62e79b513" (UID: "38bc129a-15c0-4019-b32d-e4b62e79b513"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:50:24.217868 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.217846    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38bc129a-15c0-4019-b32d-e4b62e79b513-istio-token" pod "38bc129a-15c0-4019-b32d-e4b62e79b513" (UID: "38bc129a-15c0-4019-b32d-e4b62e79b513"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:50:24.217958 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.217933    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38bc129a-15c0-4019-b32d-e4b62e79b513-istio-envoy" pod "38bc129a-15c0-4019-b32d-e4b62e79b513" (UID: "38bc129a-15c0-4019-b32d-e4b62e79b513"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:24.217958 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.217939    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38bc129a-15c0-4019-b32d-e4b62e79b513-kube-api-access-qzlf6" pod "38bc129a-15c0-4019-b32d-e4b62e79b513" (UID: "38bc129a-15c0-4019-b32d-e4b62e79b513"). InnerVolumeSpecName "kube-api-access-qzlf6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:50:24.317870 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.317827    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38bc129a-15c0-4019-b32d-e4b62e79b513-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:24.317870 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.317863    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38bc129a-15c0-4019-b32d-e4b62e79b513-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:24.317870 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.317873    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qzlf6\" (UniqueName: \"kubernetes.io/projected/38bc129a-15c0-4019-b32d-e4b62e79b513-kube-api-access-qzlf6\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:24.317870 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.317884    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38bc129a-15c0-4019-b32d-e4b62e79b513-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:24.640502 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.640462    4871 generic.go:356] "Generic (PLEG): container finished" podID="38bc129a-15c0-4019-b32d-e4b62e79b513" containerID="6fe5f2325e0c3a0a3e1f738b5e4abddc17cc27e8f7dad3e9a579f116b7455739" exitCode=0
Oct 30 22:50:24.640732 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.640523    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"
Oct 30 22:50:24.640732 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.640540    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm" event={"ID":"38bc129a-15c0-4019-b32d-e4b62e79b513","Type":"ContainerDied","Data":"6fe5f2325e0c3a0a3e1f738b5e4abddc17cc27e8f7dad3e9a579f116b7455739"}
Oct 30 22:50:24.640732 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.640589    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm" event={"ID":"38bc129a-15c0-4019-b32d-e4b62e79b513","Type":"ContainerDied","Data":"124792a0cd3f99548824faa7cc6d5e6e6081a7dbbe7dd52744332f609bda5b2e"}
Oct 30 22:50:24.640732 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.640610    4871 scope.go:122] "RemoveContainer" containerID="6fe5f2325e0c3a0a3e1f738b5e4abddc17cc27e8f7dad3e9a579f116b7455739"
Oct 30 22:50:24.653445 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.653345    4871 scope.go:122] "RemoveContainer" containerID="b94fbf535e2488f0a0a0edf651ee20c0a96d29d6a32132410586abaeaddf90d1"
Oct 30 22:50:24.664548 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.664524    4871 scope.go:122] "RemoveContainer" containerID="6fe5f2325e0c3a0a3e1f738b5e4abddc17cc27e8f7dad3e9a579f116b7455739"
Oct 30 22:50:24.664906 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:50:24.664884    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fe5f2325e0c3a0a3e1f738b5e4abddc17cc27e8f7dad3e9a579f116b7455739\": container with ID starting with 6fe5f2325e0c3a0a3e1f738b5e4abddc17cc27e8f7dad3e9a579f116b7455739 not found: ID does not exist" containerID="6fe5f2325e0c3a0a3e1f738b5e4abddc17cc27e8f7dad3e9a579f116b7455739"
Oct 30 22:50:24.664966 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.664916    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fe5f2325e0c3a0a3e1f738b5e4abddc17cc27e8f7dad3e9a579f116b7455739"} err="failed to get container status \"6fe5f2325e0c3a0a3e1f738b5e4abddc17cc27e8f7dad3e9a579f116b7455739\": rpc error: code = NotFound desc = could not find container \"6fe5f2325e0c3a0a3e1f738b5e4abddc17cc27e8f7dad3e9a579f116b7455739\": container with ID starting with 6fe5f2325e0c3a0a3e1f738b5e4abddc17cc27e8f7dad3e9a579f116b7455739 not found: ID does not exist"
Oct 30 22:50:24.664966 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.664934    4871 scope.go:122] "RemoveContainer" containerID="b94fbf535e2488f0a0a0edf651ee20c0a96d29d6a32132410586abaeaddf90d1"
Oct 30 22:50:24.665260 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:50:24.665219    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b94fbf535e2488f0a0a0edf651ee20c0a96d29d6a32132410586abaeaddf90d1\": container with ID starting with b94fbf535e2488f0a0a0edf651ee20c0a96d29d6a32132410586abaeaddf90d1 not found: ID does not exist" containerID="b94fbf535e2488f0a0a0edf651ee20c0a96d29d6a32132410586abaeaddf90d1"
Oct 30 22:50:24.665324 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.665256    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b94fbf535e2488f0a0a0edf651ee20c0a96d29d6a32132410586abaeaddf90d1"} err="failed to get container status \"b94fbf535e2488f0a0a0edf651ee20c0a96d29d6a32132410586abaeaddf90d1\": rpc error: code = NotFound desc = could not find container \"b94fbf535e2488f0a0a0edf651ee20c0a96d29d6a32132410586abaeaddf90d1\": container with ID starting with b94fbf535e2488f0a0a0edf651ee20c0a96d29d6a32132410586abaeaddf90d1 not found: ID does not exist"
Oct 30 22:50:24.814656 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.814617    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"]
Oct 30 22:50:24.866916 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:24.866873    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-6b46c3-openshift-default-56b64694fc-8lczm"]
Oct 30 22:50:25.960438 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:25.960399    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="38bc129a-15c0-4019-b32d-e4b62e79b513" path="/var/lib/kubelet/pods/38bc129a-15c0-4019-b32d-e4b62e79b513/volumes"
Oct 30 22:50:26.748473 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:26.748433    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:27.324560 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:27.324514    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-26743a-fcb59c9c7-s4rnl" podUID="ff4cdc4a-dff3-4e33-beae-ce10bb079a80" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:50:27.325045 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:27.324514    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-26743a-75976bb55f-5vl8h" podUID="9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:50:27.774699 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:27.774666    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:28.001479 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:28.001426    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:28.182856 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:28.182810    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:31.109949 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:31.109917    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:31.312196 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:31.312159    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:31.978168 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:31.978135    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:32.258476 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:32.258388    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-43d592-6cf85b6645-f4dvl"]
Oct 30 22:50:32.373204 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:32.373173    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-43d592-54c9b96dcf-6bwjj"]
Oct 30 22:50:32.698756 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:32.698720    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:32.861774 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:32.861741    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:34.820622 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:34.820578    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:34.972565 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:34.972532    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:35.571080 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:50:35.571041    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:50:36.638413 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:36.638376    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:37.053726 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:37.053697    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-26743a-fcb59c9c7-s4rnl"
Oct 30 22:50:37.097670 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:37.097629    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ff4cdc4a-dff3-4e33-beae-ce10bb079a80-kube-api-access-l5hbc\" (UniqueName: \"kubernetes.io/projected/ff4cdc4a-dff3-4e33-beae-ce10bb079a80-kube-api-access-l5hbc\") pod \"ff4cdc4a-dff3-4e33-beae-ce10bb079a80\" (UID: \"ff4cdc4a-dff3-4e33-beae-ce10bb079a80\") "
Oct 30 22:50:37.097894 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:37.097873    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ff4cdc4a-dff3-4e33-beae-ce10bb079a80-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff4cdc4a-dff3-4e33-beae-ce10bb079a80-plugins-config-volume\") pod \"ff4cdc4a-dff3-4e33-beae-ce10bb079a80\" (UID: \"ff4cdc4a-dff3-4e33-beae-ce10bb079a80\") "
Oct 30 22:50:37.098316 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:37.098286    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff4cdc4a-dff3-4e33-beae-ce10bb079a80-plugins-config-volume" pod "ff4cdc4a-dff3-4e33-beae-ce10bb079a80" (UID: "ff4cdc4a-dff3-4e33-beae-ce10bb079a80"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:50:37.098476 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:37.098448    4871 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff4cdc4a-dff3-4e33-beae-ce10bb079a80-plugins-config-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:37.099956 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:37.099923    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff4cdc4a-dff3-4e33-beae-ce10bb079a80-kube-api-access-l5hbc" pod "ff4cdc4a-dff3-4e33-beae-ce10bb079a80" (UID: "ff4cdc4a-dff3-4e33-beae-ce10bb079a80"). InnerVolumeSpecName "kube-api-access-l5hbc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:50:37.199884 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:37.199849    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l5hbc\" (UniqueName: \"kubernetes.io/projected/ff4cdc4a-dff3-4e33-beae-ce10bb079a80-kube-api-access-l5hbc\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:37.213627 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:37.213594    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-26743a-75976bb55f-5vl8h"
Oct 30 22:50:37.259563 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:37.259511    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-43d592-6cf85b6645-f4dvl" podUID="3ab9ef68-5a6d-495f-be71-68366e8ab27d" containerName="payload-processing" containerID="cri-o://582e68a8923e599b6b0503f83367e54abb24ba48d9d41d13a8849fc1485a6f90" gracePeriod=25
Oct 30 22:50:37.300828 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:37.300781    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c-plugins-config-volume\") pod \"9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c\" (UID: \"9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c\") "
Oct 30 22:50:37.301014 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:37.300949    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c-kube-api-access-vcqtm\" (UniqueName: \"kubernetes.io/projected/9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c-kube-api-access-vcqtm\") pod \"9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c\" (UID: \"9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c\") "
Oct 30 22:50:37.301275 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:37.301231    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c-plugins-config-volume" pod "9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c" (UID: "9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:50:37.301406 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:37.301387    4871 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c-plugins-config-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:37.303054 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:37.303032    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c-kube-api-access-vcqtm" pod "9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c" (UID: "9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c"). InnerVolumeSpecName "kube-api-access-vcqtm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:50:37.373642 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:37.373594    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-43d592-54c9b96dcf-6bwjj" podUID="0c63b14b-1649-4c30-b788-81e4e75e40f4" containerName="payload-pre-processing" containerID="cri-o://67ef3c2eb139be350d421fa060a4ad9dee505758429b6b98f847e244052d5bd2" gracePeriod=25
Oct 30 22:50:37.402606 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:37.402564    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vcqtm\" (UniqueName: \"kubernetes.io/projected/9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c-kube-api-access-vcqtm\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:37.797103 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:37.797063    4871 generic.go:356] "Generic (PLEG): container finished" podID="ff4cdc4a-dff3-4e33-beae-ce10bb079a80" containerID="5e9e0f59cacf0089232321e2abaae89cf0c298a82dcdb0da7aee0c1b0239b8de" exitCode=0
Oct 30 22:50:37.797671 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:37.797128    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-26743a-fcb59c9c7-s4rnl"
Oct 30 22:50:37.797671 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:37.797146    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-26743a-fcb59c9c7-s4rnl" event={"ID":"ff4cdc4a-dff3-4e33-beae-ce10bb079a80","Type":"ContainerDied","Data":"5e9e0f59cacf0089232321e2abaae89cf0c298a82dcdb0da7aee0c1b0239b8de"}
Oct 30 22:50:37.797671 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:37.797206    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-26743a-fcb59c9c7-s4rnl" event={"ID":"ff4cdc4a-dff3-4e33-beae-ce10bb079a80","Type":"ContainerDied","Data":"83df7e028de913b5109e4b50f36f52c134ab34a09588db832c96ab2f320cc813"}
Oct 30 22:50:37.797671 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:37.797257    4871 scope.go:122] "RemoveContainer" containerID="5e9e0f59cacf0089232321e2abaae89cf0c298a82dcdb0da7aee0c1b0239b8de"
Oct 30 22:50:37.798810 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:37.798783    4871 generic.go:356] "Generic (PLEG): container finished" podID="9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c" containerID="ebceb8b74027404ef3569a82d81e803020729f8925c67ec748db13d39979a7ce" exitCode=0
Oct 30 22:50:37.798902 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:37.798837    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-26743a-75976bb55f-5vl8h"
Oct 30 22:50:37.798902 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:37.798855    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-26743a-75976bb55f-5vl8h" event={"ID":"9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c","Type":"ContainerDied","Data":"ebceb8b74027404ef3569a82d81e803020729f8925c67ec748db13d39979a7ce"}
Oct 30 22:50:37.798902 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:37.798896    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-26743a-75976bb55f-5vl8h" event={"ID":"9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c","Type":"ContainerDied","Data":"90fbfd425060d773d7f22145d76d6336d63131cf4588a47da09c0097e1db1cca"}
Oct 30 22:50:37.820199 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:37.820036    4871 scope.go:122] "RemoveContainer" containerID="5e9e0f59cacf0089232321e2abaae89cf0c298a82dcdb0da7aee0c1b0239b8de"
Oct 30 22:50:37.820520 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:50:37.820497    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e9e0f59cacf0089232321e2abaae89cf0c298a82dcdb0da7aee0c1b0239b8de\": container with ID starting with 5e9e0f59cacf0089232321e2abaae89cf0c298a82dcdb0da7aee0c1b0239b8de not found: ID does not exist" containerID="5e9e0f59cacf0089232321e2abaae89cf0c298a82dcdb0da7aee0c1b0239b8de"
Oct 30 22:50:37.820563 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:37.820529    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e9e0f59cacf0089232321e2abaae89cf0c298a82dcdb0da7aee0c1b0239b8de"} err="failed to get container status \"5e9e0f59cacf0089232321e2abaae89cf0c298a82dcdb0da7aee0c1b0239b8de\": rpc error: code = NotFound desc = could not find container \"5e9e0f59cacf0089232321e2abaae89cf0c298a82dcdb0da7aee0c1b0239b8de\": container with ID starting with 5e9e0f59cacf0089232321e2abaae89cf0c298a82dcdb0da7aee0c1b0239b8de not found: ID does not exist"
Oct 30 22:50:37.820563 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:37.820547    4871 scope.go:122] "RemoveContainer" containerID="ebceb8b74027404ef3569a82d81e803020729f8925c67ec748db13d39979a7ce"
Oct 30 22:50:37.830544 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:37.830521    4871 scope.go:122] "RemoveContainer" containerID="ebceb8b74027404ef3569a82d81e803020729f8925c67ec748db13d39979a7ce"
Oct 30 22:50:37.830877 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:50:37.830852    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebceb8b74027404ef3569a82d81e803020729f8925c67ec748db13d39979a7ce\": container with ID starting with ebceb8b74027404ef3569a82d81e803020729f8925c67ec748db13d39979a7ce not found: ID does not exist" containerID="ebceb8b74027404ef3569a82d81e803020729f8925c67ec748db13d39979a7ce"
Oct 30 22:50:37.830985 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:37.830885    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebceb8b74027404ef3569a82d81e803020729f8925c67ec748db13d39979a7ce"} err="failed to get container status \"ebceb8b74027404ef3569a82d81e803020729f8925c67ec748db13d39979a7ce\": rpc error: code = NotFound desc = could not find container \"ebceb8b74027404ef3569a82d81e803020729f8925c67ec748db13d39979a7ce\": container with ID starting with ebceb8b74027404ef3569a82d81e803020729f8925c67ec748db13d39979a7ce not found: ID does not exist"
Oct 30 22:50:37.971962 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:37.971925    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-26743a-fcb59c9c7-s4rnl"]
Oct 30 22:50:38.025688 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:38.025642    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-26743a-fcb59c9c7-s4rnl"]
Oct 30 22:50:38.140666 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:38.140626    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-26743a-75976bb55f-5vl8h"]
Oct 30 22:50:38.150809 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:38.150764    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-26743a-75976bb55f-5vl8h"]
Oct 30 22:50:39.960568 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:39.960535    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c" path="/var/lib/kubelet/pods/9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c/volumes"
Oct 30 22:50:39.961022 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:39.961007    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ff4cdc4a-dff3-4e33-beae-ce10bb079a80" path="/var/lib/kubelet/pods/ff4cdc4a-dff3-4e33-beae-ce10bb079a80/volumes"
Oct 30 22:50:41.190973 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:41.190928    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-43d592-54c9b96dcf-6bwjj" podUID="0c63b14b-1649-4c30-b788-81e4e75e40f4" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:50:41.191398 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:41.190928    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-43d592-6cf85b6645-f4dvl" podUID="3ab9ef68-5a6d-495f-be71-68366e8ab27d" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:50:41.229691 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:41.229656    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:41.575988 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:41.575932    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:41.740195 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:41.740089    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:43.268787 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:43.268755    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:45.930964 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:45.930927    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:49.123425 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:49.123385    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:49.282461 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:49.282424    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:50.529932 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:50.529891    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:51.191223 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:51.191178    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-43d592-6cf85b6645-f4dvl" podUID="3ab9ef68-5a6d-495f-be71-68366e8ab27d" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:50:51.191454 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:51.191186    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-43d592-54c9b96dcf-6bwjj" podUID="0c63b14b-1649-4c30-b788-81e4e75e40f4" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:50:54.127712 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:54.127677    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:54.370979 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:54.370944    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:54.415666 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:54.415572    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76"]
Oct 30 22:50:54.415882 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:54.415779    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76" podUID="f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3" containerName="maas-api" containerID="cri-o://8df201acc09b5f6df9292f6f520f92cc80134f581f7a22d04675c9a0f0767742" gracePeriod=30
Oct 30 22:50:54.607174 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:54.607125    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:54.733384 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:54.733351    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76"
Oct 30 22:50:54.747521 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:54.747489    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3-maas-api-tls\") pod \"f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3\" (UID: \"f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3\") "
Oct 30 22:50:54.747731 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:54.747581    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3-openshift-service-ca\") pod \"f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3\" (UID: \"f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3\") "
Oct 30 22:50:54.747731 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:54.747631    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3-kube-api-access-n8l6v\" (UniqueName: \"kubernetes.io/projected/f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3-kube-api-access-n8l6v\") pod \"f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3\" (UID: \"f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3\") "
Oct 30 22:50:54.747731 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:54.747656    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3-metrics-certs\") pod \"f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3\" (UID: \"f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3\") "
Oct 30 22:50:54.748057 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:54.748022    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3-openshift-service-ca" pod "f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3" (UID: "f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:50:54.749856 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:54.749829    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3-kube-api-access-n8l6v" pod "f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3" (UID: "f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3"). InnerVolumeSpecName "kube-api-access-n8l6v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:50:54.749970 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:54.749864    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3-maas-api-tls" pod "f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3" (UID: "f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:50:54.749970 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:54.749916    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3-metrics-certs" pod "f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3" (UID: "f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:50:54.848765 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:54.848720    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3-maas-api-tls\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:54.848765 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:54.848753    4871 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3-openshift-service-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:54.848765 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:54.848768    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n8l6v\" (UniqueName: \"kubernetes.io/projected/f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3-kube-api-access-n8l6v\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:54.848765 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:54.848778    4871 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3-metrics-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:54.934162 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:54.934127    4871 generic.go:356] "Generic (PLEG): container finished" podID="f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3" containerID="8df201acc09b5f6df9292f6f520f92cc80134f581f7a22d04675c9a0f0767742" exitCode=0
Oct 30 22:50:54.934394 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:54.934183    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76"
Oct 30 22:50:54.934394 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:54.934179    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76" event={"ID":"f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3","Type":"ContainerDied","Data":"8df201acc09b5f6df9292f6f520f92cc80134f581f7a22d04675c9a0f0767742"}
Oct 30 22:50:54.934394 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:54.934335    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76" event={"ID":"f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3","Type":"ContainerDied","Data":"ac13a7d74b1c5618bc409a8a137e2e51cb5d0b0e01eaeee22170680274d9d806"}
Oct 30 22:50:54.934394 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:54.934365    4871 scope.go:122] "RemoveContainer" containerID="8df201acc09b5f6df9292f6f520f92cc80134f581f7a22d04675c9a0f0767742"
Oct 30 22:50:54.947170 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:54.947141    4871 scope.go:122] "RemoveContainer" containerID="8df201acc09b5f6df9292f6f520f92cc80134f581f7a22d04675c9a0f0767742"
Oct 30 22:50:54.947556 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:50:54.947538    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8df201acc09b5f6df9292f6f520f92cc80134f581f7a22d04675c9a0f0767742\": container with ID starting with 8df201acc09b5f6df9292f6f520f92cc80134f581f7a22d04675c9a0f0767742 not found: ID does not exist" containerID="8df201acc09b5f6df9292f6f520f92cc80134f581f7a22d04675c9a0f0767742"
Oct 30 22:50:54.947622 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:54.947565    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8df201acc09b5f6df9292f6f520f92cc80134f581f7a22d04675c9a0f0767742"} err="failed to get container status \"8df201acc09b5f6df9292f6f520f92cc80134f581f7a22d04675c9a0f0767742\": rpc error: code = NotFound desc = could not find container \"8df201acc09b5f6df9292f6f520f92cc80134f581f7a22d04675c9a0f0767742\": container with ID starting with 8df201acc09b5f6df9292f6f520f92cc80134f581f7a22d04675c9a0f0767742 not found: ID does not exist"
Oct 30 22:50:55.108462 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:55.108368    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76"]
Oct 30 22:50:55.112675 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:55.112643    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5e884c-58bb45f5bf-cxj76"]
Oct 30 22:50:55.585843 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:55.585808    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:55.962929 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:55.962893    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3" path="/var/lib/kubelet/pods/f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3/volumes"
Oct 30 22:50:56.021118 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:56.021075    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:56.245840 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:56.245746    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:56.403480 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:56.403448    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:56.920459 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:56.920429    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:57.605713 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:57.605608    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-43d592-6cf85b6645-f4dvl"
Oct 30 22:50:57.690157 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:57.690126    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3ab9ef68-5a6d-495f-be71-68366e8ab27d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ab9ef68-5a6d-495f-be71-68366e8ab27d-plugins-config-volume\") pod \"3ab9ef68-5a6d-495f-be71-68366e8ab27d\" (UID: \"3ab9ef68-5a6d-495f-be71-68366e8ab27d\") "
Oct 30 22:50:57.690361 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:57.690232    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3ab9ef68-5a6d-495f-be71-68366e8ab27d-kube-api-access-994tf\" (UniqueName: \"kubernetes.io/projected/3ab9ef68-5a6d-495f-be71-68366e8ab27d-kube-api-access-994tf\") pod \"3ab9ef68-5a6d-495f-be71-68366e8ab27d\" (UID: \"3ab9ef68-5a6d-495f-be71-68366e8ab27d\") "
Oct 30 22:50:57.690847 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:57.690650    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ab9ef68-5a6d-495f-be71-68366e8ab27d-plugins-config-volume" pod "3ab9ef68-5a6d-495f-be71-68366e8ab27d" (UID: "3ab9ef68-5a6d-495f-be71-68366e8ab27d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:50:57.690847 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:57.690736    4871 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ab9ef68-5a6d-495f-be71-68366e8ab27d-plugins-config-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:57.692830 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:57.692797    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ab9ef68-5a6d-495f-be71-68366e8ab27d-kube-api-access-994tf" pod "3ab9ef68-5a6d-495f-be71-68366e8ab27d" (UID: "3ab9ef68-5a6d-495f-be71-68366e8ab27d"). InnerVolumeSpecName "kube-api-access-994tf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:50:57.694664 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:57.694644    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-43d592-54c9b96dcf-6bwjj"
Oct 30 22:50:57.791796 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:57.791746    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0c63b14b-1649-4c30-b788-81e4e75e40f4-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c63b14b-1649-4c30-b788-81e4e75e40f4-plugins-config-volume\") pod \"0c63b14b-1649-4c30-b788-81e4e75e40f4\" (UID: \"0c63b14b-1649-4c30-b788-81e4e75e40f4\") "
Oct 30 22:50:57.792040 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:57.792005    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0c63b14b-1649-4c30-b788-81e4e75e40f4-kube-api-access-h8gzw\" (UniqueName: \"kubernetes.io/projected/0c63b14b-1649-4c30-b788-81e4e75e40f4-kube-api-access-h8gzw\") pod \"0c63b14b-1649-4c30-b788-81e4e75e40f4\" (UID: \"0c63b14b-1649-4c30-b788-81e4e75e40f4\") "
Oct 30 22:50:57.792574 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:57.792547    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c63b14b-1649-4c30-b788-81e4e75e40f4-plugins-config-volume" pod "0c63b14b-1649-4c30-b788-81e4e75e40f4" (UID: "0c63b14b-1649-4c30-b788-81e4e75e40f4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:50:57.792703 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:57.792569    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-994tf\" (UniqueName: \"kubernetes.io/projected/3ab9ef68-5a6d-495f-be71-68366e8ab27d-kube-api-access-994tf\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:57.795154 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:57.795107    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c63b14b-1649-4c30-b788-81e4e75e40f4-kube-api-access-h8gzw" pod "0c63b14b-1649-4c30-b788-81e4e75e40f4" (UID: "0c63b14b-1649-4c30-b788-81e4e75e40f4"). InnerVolumeSpecName "kube-api-access-h8gzw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:50:57.893724 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:57.893680    4871 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c63b14b-1649-4c30-b788-81e4e75e40f4-plugins-config-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:57.893724 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:57.893724    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8gzw\" (UniqueName: \"kubernetes.io/projected/0c63b14b-1649-4c30-b788-81e4e75e40f4-kube-api-access-h8gzw\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:57.961142 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:57.961097    4871 generic.go:356] "Generic (PLEG): container finished" podID="0c63b14b-1649-4c30-b788-81e4e75e40f4" containerID="67ef3c2eb139be350d421fa060a4ad9dee505758429b6b98f847e244052d5bd2" exitCode=0
Oct 30 22:50:57.961690 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:57.961223    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-43d592-54c9b96dcf-6bwjj"
Oct 30 22:50:57.962913 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:57.962883    4871 generic.go:356] "Generic (PLEG): container finished" podID="3ab9ef68-5a6d-495f-be71-68366e8ab27d" containerID="582e68a8923e599b6b0503f83367e54abb24ba48d9d41d13a8849fc1485a6f90" exitCode=0
Oct 30 22:50:57.963075 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:57.962994    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-43d592-6cf85b6645-f4dvl"
Oct 30 22:50:57.964533 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:57.964501    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-43d592-54c9b96dcf-6bwjj" event={"ID":"0c63b14b-1649-4c30-b788-81e4e75e40f4","Type":"ContainerDied","Data":"67ef3c2eb139be350d421fa060a4ad9dee505758429b6b98f847e244052d5bd2"}
Oct 30 22:50:57.964838 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:57.964547    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-43d592-54c9b96dcf-6bwjj" event={"ID":"0c63b14b-1649-4c30-b788-81e4e75e40f4","Type":"ContainerDied","Data":"28d32bc50dc2b50132d8fe2d9436125c60ba5ec99975ed8829073d26c95c17f5"}
Oct 30 22:50:57.964838 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:57.964573    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-43d592-6cf85b6645-f4dvl" event={"ID":"3ab9ef68-5a6d-495f-be71-68366e8ab27d","Type":"ContainerDied","Data":"582e68a8923e599b6b0503f83367e54abb24ba48d9d41d13a8849fc1485a6f90"}
Oct 30 22:50:57.964838 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:57.964594    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-43d592-6cf85b6645-f4dvl" event={"ID":"3ab9ef68-5a6d-495f-be71-68366e8ab27d","Type":"ContainerDied","Data":"99653443e44ccfc8c60e4b8628303f7b83c4df476ac1ae56b83e7e28b4e0b592"}
Oct 30 22:50:57.964838 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:57.964640    4871 scope.go:122] "RemoveContainer" containerID="67ef3c2eb139be350d421fa060a4ad9dee505758429b6b98f847e244052d5bd2"
Oct 30 22:50:57.984115 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:57.984091    4871 scope.go:122] "RemoveContainer" containerID="67ef3c2eb139be350d421fa060a4ad9dee505758429b6b98f847e244052d5bd2"
Oct 30 22:50:57.984518 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:50:57.984493    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67ef3c2eb139be350d421fa060a4ad9dee505758429b6b98f847e244052d5bd2\": container with ID starting with 67ef3c2eb139be350d421fa060a4ad9dee505758429b6b98f847e244052d5bd2 not found: ID does not exist" containerID="67ef3c2eb139be350d421fa060a4ad9dee505758429b6b98f847e244052d5bd2"
Oct 30 22:50:57.984588 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:57.984530    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67ef3c2eb139be350d421fa060a4ad9dee505758429b6b98f847e244052d5bd2"} err="failed to get container status \"67ef3c2eb139be350d421fa060a4ad9dee505758429b6b98f847e244052d5bd2\": rpc error: code = NotFound desc = could not find container \"67ef3c2eb139be350d421fa060a4ad9dee505758429b6b98f847e244052d5bd2\": container with ID starting with 67ef3c2eb139be350d421fa060a4ad9dee505758429b6b98f847e244052d5bd2 not found: ID does not exist"
Oct 30 22:50:57.984588 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:57.984555    4871 scope.go:122] "RemoveContainer" containerID="582e68a8923e599b6b0503f83367e54abb24ba48d9d41d13a8849fc1485a6f90"
Oct 30 22:50:57.996098 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:57.996068    4871 scope.go:122] "RemoveContainer" containerID="582e68a8923e599b6b0503f83367e54abb24ba48d9d41d13a8849fc1485a6f90"
Oct 30 22:50:57.996485 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:50:57.996453    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"582e68a8923e599b6b0503f83367e54abb24ba48d9d41d13a8849fc1485a6f90\": container with ID starting with 582e68a8923e599b6b0503f83367e54abb24ba48d9d41d13a8849fc1485a6f90 not found: ID does not exist" containerID="582e68a8923e599b6b0503f83367e54abb24ba48d9d41d13a8849fc1485a6f90"
Oct 30 22:50:57.996605 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:57.996492    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"582e68a8923e599b6b0503f83367e54abb24ba48d9d41d13a8849fc1485a6f90"} err="failed to get container status \"582e68a8923e599b6b0503f83367e54abb24ba48d9d41d13a8849fc1485a6f90\": rpc error: code = NotFound desc = could not find container \"582e68a8923e599b6b0503f83367e54abb24ba48d9d41d13a8849fc1485a6f90\": container with ID starting with 582e68a8923e599b6b0503f83367e54abb24ba48d9d41d13a8849fc1485a6f90 not found: ID does not exist"
Oct 30 22:50:58.137114 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:58.137076    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-43d592-54c9b96dcf-6bwjj"]
Oct 30 22:50:58.141044 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:58.141012    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-43d592-54c9b96dcf-6bwjj"]
Oct 30 22:50:58.258106 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:58.257997    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-43d592-6cf85b6645-f4dvl"]
Oct 30 22:50:58.262954 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:58.262924    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-43d592-6cf85b6645-f4dvl"]
Oct 30 22:50:59.960701 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:59.960666    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0c63b14b-1649-4c30-b788-81e4e75e40f4" path="/var/lib/kubelet/pods/0c63b14b-1649-4c30-b788-81e4e75e40f4/volumes"
Oct 30 22:50:59.961128 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:50:59.961089    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3ab9ef68-5a6d-495f-be71-68366e8ab27d" path="/var/lib/kubelet/pods/3ab9ef68-5a6d-495f-be71-68366e8ab27d/volumes"
Oct 30 22:51:00.551144 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:00.551103    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64"]
Oct 30 22:51:00.551413 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:00.551386    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64" podUID="3ac0939d-60c7-400b-b345-4acb5def8620" containerName="maas-api" containerID="cri-o://a247084742c0a20660f4cecd48d74eb2fa392d43647679d6aa8500fd0600d9cc" gracePeriod=30
Oct 30 22:51:00.648070 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:00.648040    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:00.871698 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:00.871671    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64"
Oct 30 22:51:00.929370 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:00.929335    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3ac0939d-60c7-400b-b345-4acb5def8620-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ac0939d-60c7-400b-b345-4acb5def8620-openshift-service-ca\") pod \"3ac0939d-60c7-400b-b345-4acb5def8620\" (UID: \"3ac0939d-60c7-400b-b345-4acb5def8620\") "
Oct 30 22:51:00.929594 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:00.929432    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3ac0939d-60c7-400b-b345-4acb5def8620-kube-api-access-5ftsq\" (UniqueName: \"kubernetes.io/projected/3ac0939d-60c7-400b-b345-4acb5def8620-kube-api-access-5ftsq\") pod \"3ac0939d-60c7-400b-b345-4acb5def8620\" (UID: \"3ac0939d-60c7-400b-b345-4acb5def8620\") "
Oct 30 22:51:00.929678 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:00.929593    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3ac0939d-60c7-400b-b345-4acb5def8620-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ac0939d-60c7-400b-b345-4acb5def8620-metrics-certs\") pod \"3ac0939d-60c7-400b-b345-4acb5def8620\" (UID: \"3ac0939d-60c7-400b-b345-4acb5def8620\") "
Oct 30 22:51:00.929678 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:00.929647    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3ac0939d-60c7-400b-b345-4acb5def8620-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ac0939d-60c7-400b-b345-4acb5def8620-maas-api-tls\") pod \"3ac0939d-60c7-400b-b345-4acb5def8620\" (UID: \"3ac0939d-60c7-400b-b345-4acb5def8620\") "
Oct 30 22:51:00.930061 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:00.929793    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ac0939d-60c7-400b-b345-4acb5def8620-openshift-service-ca" pod "3ac0939d-60c7-400b-b345-4acb5def8620" (UID: "3ac0939d-60c7-400b-b345-4acb5def8620"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:51:00.941859 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:00.932949    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ac0939d-60c7-400b-b345-4acb5def8620-maas-api-tls" pod "3ac0939d-60c7-400b-b345-4acb5def8620" (UID: "3ac0939d-60c7-400b-b345-4acb5def8620"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:51:00.941859 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:00.932984    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ac0939d-60c7-400b-b345-4acb5def8620-metrics-certs" pod "3ac0939d-60c7-400b-b345-4acb5def8620" (UID: "3ac0939d-60c7-400b-b345-4acb5def8620"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:51:00.941859 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:00.933736    4871 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ac0939d-60c7-400b-b345-4acb5def8620-openshift-service-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:00.941859 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:00.939562    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ac0939d-60c7-400b-b345-4acb5def8620-kube-api-access-5ftsq" pod "3ac0939d-60c7-400b-b345-4acb5def8620" (UID: "3ac0939d-60c7-400b-b345-4acb5def8620"). InnerVolumeSpecName "kube-api-access-5ftsq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:51:01.026279 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:01.018639    4871 generic.go:356] "Generic (PLEG): container finished" podID="3ac0939d-60c7-400b-b345-4acb5def8620" containerID="a247084742c0a20660f4cecd48d74eb2fa392d43647679d6aa8500fd0600d9cc" exitCode=0
Oct 30 22:51:01.026279 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:01.018766    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64"
Oct 30 22:51:01.026279 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:01.018782    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64" event={"ID":"3ac0939d-60c7-400b-b345-4acb5def8620","Type":"ContainerDied","Data":"a247084742c0a20660f4cecd48d74eb2fa392d43647679d6aa8500fd0600d9cc"}
Oct 30 22:51:01.026279 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:01.018847    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64" event={"ID":"3ac0939d-60c7-400b-b345-4acb5def8620","Type":"ContainerDied","Data":"2e693baa099c8228d686e791146deb0df856f026c446adf37d1f44294fa4c210"}
Oct 30 22:51:01.026279 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:01.018877    4871 scope.go:122] "RemoveContainer" containerID="a247084742c0a20660f4cecd48d74eb2fa392d43647679d6aa8500fd0600d9cc"
Oct 30 22:51:01.035357 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:01.035328    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ftsq\" (UniqueName: \"kubernetes.io/projected/3ac0939d-60c7-400b-b345-4acb5def8620-kube-api-access-5ftsq\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:01.035357 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:01.035354    4871 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ac0939d-60c7-400b-b345-4acb5def8620-metrics-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:01.035589 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:01.035370    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ac0939d-60c7-400b-b345-4acb5def8620-maas-api-tls\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:01.035882 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:01.035862    4871 scope.go:122] "RemoveContainer" containerID="a247084742c0a20660f4cecd48d74eb2fa392d43647679d6aa8500fd0600d9cc"
Oct 30 22:51:01.036193 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:51:01.036171    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a247084742c0a20660f4cecd48d74eb2fa392d43647679d6aa8500fd0600d9cc\": container with ID starting with a247084742c0a20660f4cecd48d74eb2fa392d43647679d6aa8500fd0600d9cc not found: ID does not exist" containerID="a247084742c0a20660f4cecd48d74eb2fa392d43647679d6aa8500fd0600d9cc"
Oct 30 22:51:01.036305 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:01.036201    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a247084742c0a20660f4cecd48d74eb2fa392d43647679d6aa8500fd0600d9cc"} err="failed to get container status \"a247084742c0a20660f4cecd48d74eb2fa392d43647679d6aa8500fd0600d9cc\": rpc error: code = NotFound desc = could not find container \"a247084742c0a20660f4cecd48d74eb2fa392d43647679d6aa8500fd0600d9cc\": container with ID starting with a247084742c0a20660f4cecd48d74eb2fa392d43647679d6aa8500fd0600d9cc not found: ID does not exist"
Oct 30 22:51:01.193626 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:01.193593    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64"]
Oct 30 22:51:01.199290 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:01.199257    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-26743a-56f465b6f4-2vl64"]
Oct 30 22:51:01.960294 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:01.960260    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3ac0939d-60c7-400b-b345-4acb5def8620" path="/var/lib/kubelet/pods/3ac0939d-60c7-400b-b345-4acb5def8620/volumes"
Oct 30 22:51:02.364315 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:02.364284    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:02.520577 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:02.520541    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:02.718765 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:02.718677    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:03.323883 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:03.323843    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:03.630834 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:03.630800    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:03.783030 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:03.782995    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:03.974876 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:03.974772    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"]
Oct 30 22:51:03.975085 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:03.974998    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x" podUID="ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea" containerName="istio-proxy" containerID="cri-o://a7d1b0220ae04364eacc4053d39b06b2ef39977da2a77ac4708ec91e40e2a431" gracePeriod=30
Oct 30 22:51:03.983699 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:03.983674    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:04.261163 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261069    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn"]
Oct 30 22:51:04.261710 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261689    4871 cpu_manager.go:414] "containerMap: removing container" podUID="93c5282f-81b6-4518-b673-68f527fdfd04" containerName="main"
Oct 30 22:51:04.261710 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261710    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="93c5282f-81b6-4518-b673-68f527fdfd04" containerName="main"
Oct 30 22:51:04.261890 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261726    4871 cpu_manager.go:414] "containerMap: removing container" podUID="9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c" containerName="payload-pre-processing"
Oct 30 22:51:04.261890 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261731    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c" containerName="payload-pre-processing"
Oct 30 22:51:04.261890 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261741    4871 cpu_manager.go:414] "containerMap: removing container" podUID="6cbecab8-bfad-40bf-850a-905e4d365d29" containerName="istio-proxy"
Oct 30 22:51:04.261890 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261748    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6cbecab8-bfad-40bf-850a-905e4d365d29" containerName="istio-proxy"
Oct 30 22:51:04.261890 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261756    4871 cpu_manager.go:414] "containerMap: removing container" podUID="0a874ed5-421c-4199-87bf-ed1997ceb64b" containerName="maas-api"
Oct 30 22:51:04.261890 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261760    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0a874ed5-421c-4199-87bf-ed1997ceb64b" containerName="maas-api"
Oct 30 22:51:04.261890 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261766    4871 cpu_manager.go:414] "containerMap: removing container" podUID="b873ef79-e292-4496-a9ec-8d2ec09f65f6" containerName="istio-proxy"
Oct 30 22:51:04.261890 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261771    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b873ef79-e292-4496-a9ec-8d2ec09f65f6" containerName="istio-proxy"
Oct 30 22:51:04.261890 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261777    4871 cpu_manager.go:414] "containerMap: removing container" podUID="4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e" containerName="istio-proxy"
Oct 30 22:51:04.261890 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261781    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e" containerName="istio-proxy"
Oct 30 22:51:04.261890 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261787    4871 cpu_manager.go:414] "containerMap: removing container" podUID="3ac0939d-60c7-400b-b345-4acb5def8620" containerName="maas-api"
Oct 30 22:51:04.261890 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261791    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3ac0939d-60c7-400b-b345-4acb5def8620" containerName="maas-api"
Oct 30 22:51:04.261890 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261799    4871 cpu_manager.go:414] "containerMap: removing container" podUID="6cbecab8-bfad-40bf-850a-905e4d365d29" containerName="istio-proxy"
Oct 30 22:51:04.261890 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261804    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6cbecab8-bfad-40bf-850a-905e4d365d29" containerName="istio-proxy"
Oct 30 22:51:04.261890 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261812    4871 cpu_manager.go:414] "containerMap: removing container" podUID="9b7e498c-c51e-4f65-8f1c-e1fb615f278c" containerName="maas-api"
Oct 30 22:51:04.261890 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261816    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9b7e498c-c51e-4f65-8f1c-e1fb615f278c" containerName="maas-api"
Oct 30 22:51:04.261890 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261822    4871 cpu_manager.go:414] "containerMap: removing container" podUID="0c63b14b-1649-4c30-b788-81e4e75e40f4" containerName="payload-pre-processing"
Oct 30 22:51:04.261890 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261826    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0c63b14b-1649-4c30-b788-81e4e75e40f4" containerName="payload-pre-processing"
Oct 30 22:51:04.261890 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261842    4871 cpu_manager.go:414] "containerMap: removing container" podUID="ff4cdc4a-dff3-4e33-beae-ce10bb079a80" containerName="payload-processing"
Oct 30 22:51:04.261890 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261846    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ff4cdc4a-dff3-4e33-beae-ce10bb079a80" containerName="payload-processing"
Oct 30 22:51:04.261890 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261853    4871 cpu_manager.go:414] "containerMap: removing container" podUID="7c957c3f-753d-4d57-b724-19fdce296860" containerName="payload-processing"
Oct 30 22:51:04.261890 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261858    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7c957c3f-753d-4d57-b724-19fdce296860" containerName="payload-processing"
Oct 30 22:51:04.261890 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261865    4871 cpu_manager.go:414] "containerMap: removing container" podUID="1cb08cf4-430f-416a-b1f8-81b5695e3f4e" containerName="istio-proxy"
Oct 30 22:51:04.261890 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261869    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1cb08cf4-430f-416a-b1f8-81b5695e3f4e" containerName="istio-proxy"
Oct 30 22:51:04.261890 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261877    4871 cpu_manager.go:414] "containerMap: removing container" podUID="3ab9ef68-5a6d-495f-be71-68366e8ab27d" containerName="payload-processing"
Oct 30 22:51:04.261890 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261881    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3ab9ef68-5a6d-495f-be71-68366e8ab27d" containerName="payload-processing"
Oct 30 22:51:04.261890 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261889    4871 cpu_manager.go:414] "containerMap: removing container" podUID="f3798606-a072-44a5-891c-4588de3c6ff1" containerName="istio-proxy"
Oct 30 22:51:04.261890 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261895    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f3798606-a072-44a5-891c-4588de3c6ff1" containerName="istio-proxy"
Oct 30 22:51:04.261890 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261904    4871 cpu_manager.go:414] "containerMap: removing container" podUID="7ea0c6d1-5765-43c8-95c1-19473cc909e2" containerName="payload-pre-processing"
Oct 30 22:51:04.261890 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261908    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7ea0c6d1-5765-43c8-95c1-19473cc909e2" containerName="payload-pre-processing"
Oct 30 22:51:04.263143 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261922    4871 cpu_manager.go:414] "containerMap: removing container" podUID="f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3" containerName="maas-api"
Oct 30 22:51:04.263143 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261927    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3" containerName="maas-api"
Oct 30 22:51:04.263143 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261933    4871 cpu_manager.go:414] "containerMap: removing container" podUID="38bc129a-15c0-4019-b32d-e4b62e79b513" containerName="istio-proxy"
Oct 30 22:51:04.263143 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261937    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="38bc129a-15c0-4019-b32d-e4b62e79b513" containerName="istio-proxy"
Oct 30 22:51:04.263143 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261945    4871 cpu_manager.go:414] "containerMap: removing container" podUID="1cb08cf4-430f-416a-b1f8-81b5695e3f4e" containerName="istio-proxy"
Oct 30 22:51:04.263143 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261949    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1cb08cf4-430f-416a-b1f8-81b5695e3f4e" containerName="istio-proxy"
Oct 30 22:51:04.263143 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261956    4871 cpu_manager.go:414] "containerMap: removing container" podUID="b873ef79-e292-4496-a9ec-8d2ec09f65f6" containerName="istio-proxy"
Oct 30 22:51:04.263143 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261961    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b873ef79-e292-4496-a9ec-8d2ec09f65f6" containerName="istio-proxy"
Oct 30 22:51:04.263143 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261967    4871 cpu_manager.go:414] "containerMap: removing container" podUID="84329f90-dc35-41dc-b593-12c556e80f73" containerName="main"
Oct 30 22:51:04.263143 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261971    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="84329f90-dc35-41dc-b593-12c556e80f73" containerName="main"
Oct 30 22:51:04.263143 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261977    4871 cpu_manager.go:414] "containerMap: removing container" podUID="2f8e8b67-0b00-4fb5-b125-865f980b97c0" containerName="maas-api"
Oct 30 22:51:04.263143 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261981    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2f8e8b67-0b00-4fb5-b125-865f980b97c0" containerName="maas-api"
Oct 30 22:51:04.263143 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261987    4871 cpu_manager.go:414] "containerMap: removing container" podUID="533efba5-8b6c-4726-b0eb-2f7b53ac5ea3" containerName="maas-api"
Oct 30 22:51:04.263143 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.261991    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="533efba5-8b6c-4726-b0eb-2f7b53ac5ea3" containerName="maas-api"
Oct 30 22:51:04.263143 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.262067    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="9dfec38a-aaa5-4884-8fb0-9bfa4efbcb6c" containerName="payload-pre-processing"
Oct 30 22:51:04.263143 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.262076    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="6cbecab8-bfad-40bf-850a-905e4d365d29" containerName="istio-proxy"
Oct 30 22:51:04.263143 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.262082    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="0a874ed5-421c-4199-87bf-ed1997ceb64b" containerName="maas-api"
Oct 30 22:51:04.263143 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.262088    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="b873ef79-e292-4496-a9ec-8d2ec09f65f6" containerName="istio-proxy"
Oct 30 22:51:04.263143 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.262093    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="4e7a9233-fd96-4bf4-8f1e-3b9b84e02a2e" containerName="istio-proxy"
Oct 30 22:51:04.263143 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.262099    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="3ac0939d-60c7-400b-b345-4acb5def8620" containerName="maas-api"
Oct 30 22:51:04.263143 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.262105    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="9b7e498c-c51e-4f65-8f1c-e1fb615f278c" containerName="maas-api"
Oct 30 22:51:04.263143 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.262110    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="0c63b14b-1649-4c30-b788-81e4e75e40f4" containerName="payload-pre-processing"
Oct 30 22:51:04.263143 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.262117    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="ff4cdc4a-dff3-4e33-beae-ce10bb079a80" containerName="payload-processing"
Oct 30 22:51:04.263143 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.262123    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="7c957c3f-753d-4d57-b724-19fdce296860" containerName="payload-processing"
Oct 30 22:51:04.263143 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.262128    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="1cb08cf4-430f-416a-b1f8-81b5695e3f4e" containerName="istio-proxy"
Oct 30 22:51:04.263143 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.262132    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="3ab9ef68-5a6d-495f-be71-68366e8ab27d" containerName="payload-processing"
Oct 30 22:51:04.263143 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.262138    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="f3798606-a072-44a5-891c-4588de3c6ff1" containerName="istio-proxy"
Oct 30 22:51:04.263143 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.262144    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="7ea0c6d1-5765-43c8-95c1-19473cc909e2" containerName="payload-pre-processing"
Oct 30 22:51:04.263143 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.262149    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="f8940f84-c4b0-4e17-9bc3-14bb1d65e1a3" containerName="maas-api"
Oct 30 22:51:04.263143 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.262154    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="38bc129a-15c0-4019-b32d-e4b62e79b513" containerName="istio-proxy"
Oct 30 22:51:04.263143 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.262159    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="1cb08cf4-430f-416a-b1f8-81b5695e3f4e" containerName="istio-proxy"
Oct 30 22:51:04.263143 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.262164    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="84329f90-dc35-41dc-b593-12c556e80f73" containerName="main"
Oct 30 22:51:04.263143 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.262169    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="2f8e8b67-0b00-4fb5-b125-865f980b97c0" containerName="maas-api"
Oct 30 22:51:04.263143 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.262176    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="533efba5-8b6c-4726-b0eb-2f7b53ac5ea3" containerName="maas-api"
Oct 30 22:51:04.263143 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.262181    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="93c5282f-81b6-4518-b673-68f527fdfd04" containerName="main"
Oct 30 22:51:04.267391 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.267367    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn"
Oct 30 22:51:04.302003 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.301969    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9d90975-336b-49e4-b165-0b9fe73cb526-istio-envoy\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn\" (UID: \"a9d90975-336b-49e4-b165-0b9fe73cb526\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn"
Oct 30 22:51:04.302176 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.302012    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9d90975-336b-49e4-b165-0b9fe73cb526-workload-certs\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn\" (UID: \"a9d90975-336b-49e4-b165-0b9fe73cb526\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn"
Oct 30 22:51:04.302176 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.302066    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9d90975-336b-49e4-b165-0b9fe73cb526-credential-socket\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn\" (UID: \"a9d90975-336b-49e4-b165-0b9fe73cb526\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn"
Oct 30 22:51:04.302176 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.302090    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9d90975-336b-49e4-b165-0b9fe73cb526-istiod-ca-cert\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn\" (UID: \"a9d90975-336b-49e4-b165-0b9fe73cb526\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn"
Oct 30 22:51:04.302443 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.302202    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9d90975-336b-49e4-b165-0b9fe73cb526-istio-data\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn\" (UID: \"a9d90975-336b-49e4-b165-0b9fe73cb526\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn"
Oct 30 22:51:04.302443 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.302333    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9d90975-336b-49e4-b165-0b9fe73cb526-istio-podinfo\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn\" (UID: \"a9d90975-336b-49e4-b165-0b9fe73cb526\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn"
Oct 30 22:51:04.302443 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.302375    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z87zg\" (UniqueName: \"kubernetes.io/projected/a9d90975-336b-49e4-b165-0b9fe73cb526-kube-api-access-z87zg\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn\" (UID: \"a9d90975-336b-49e4-b165-0b9fe73cb526\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn"
Oct 30 22:51:04.302443 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.302407    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9d90975-336b-49e4-b165-0b9fe73cb526-istio-token\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn\" (UID: \"a9d90975-336b-49e4-b165-0b9fe73cb526\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn"
Oct 30 22:51:04.302610 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.302587    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9d90975-336b-49e4-b165-0b9fe73cb526-workload-socket\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn\" (UID: \"a9d90975-336b-49e4-b165-0b9fe73cb526\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn"
Oct 30 22:51:04.322549 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.322512    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-26743a-openshift-default-dockercfg-lddtx\""
Oct 30 22:51:04.325718 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.325682    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn"]
Oct 30 22:51:04.403848 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.403815    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9d90975-336b-49e4-b165-0b9fe73cb526-workload-socket\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn\" (UID: \"a9d90975-336b-49e4-b165-0b9fe73cb526\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn"
Oct 30 22:51:04.404074 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.403882    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9d90975-336b-49e4-b165-0b9fe73cb526-istio-envoy\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn\" (UID: \"a9d90975-336b-49e4-b165-0b9fe73cb526\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn"
Oct 30 22:51:04.404074 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.403924    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9d90975-336b-49e4-b165-0b9fe73cb526-workload-certs\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn\" (UID: \"a9d90975-336b-49e4-b165-0b9fe73cb526\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn"
Oct 30 22:51:04.404259 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.404095    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9d90975-336b-49e4-b165-0b9fe73cb526-credential-socket\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn\" (UID: \"a9d90975-336b-49e4-b165-0b9fe73cb526\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn"
Oct 30 22:51:04.404259 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.404158    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9d90975-336b-49e4-b165-0b9fe73cb526-istiod-ca-cert\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn\" (UID: \"a9d90975-336b-49e4-b165-0b9fe73cb526\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn"
Oct 30 22:51:04.404259 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.404194    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9d90975-336b-49e4-b165-0b9fe73cb526-istio-data\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn\" (UID: \"a9d90975-336b-49e4-b165-0b9fe73cb526\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn"
Oct 30 22:51:04.404259 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.404254    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9d90975-336b-49e4-b165-0b9fe73cb526-istio-podinfo\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn\" (UID: \"a9d90975-336b-49e4-b165-0b9fe73cb526\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn"
Oct 30 22:51:04.404512 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.404296    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z87zg\" (UniqueName: \"kubernetes.io/projected/a9d90975-336b-49e4-b165-0b9fe73cb526-kube-api-access-z87zg\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn\" (UID: \"a9d90975-336b-49e4-b165-0b9fe73cb526\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn"
Oct 30 22:51:04.404512 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.404318    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9d90975-336b-49e4-b165-0b9fe73cb526-workload-socket\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn\" (UID: \"a9d90975-336b-49e4-b165-0b9fe73cb526\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn"
Oct 30 22:51:04.404512 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.404333    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9d90975-336b-49e4-b165-0b9fe73cb526-istio-token\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn\" (UID: \"a9d90975-336b-49e4-b165-0b9fe73cb526\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn"
Oct 30 22:51:04.404512 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.404429    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9d90975-336b-49e4-b165-0b9fe73cb526-workload-certs\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn\" (UID: \"a9d90975-336b-49e4-b165-0b9fe73cb526\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn"
Oct 30 22:51:04.405294 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.404602    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9d90975-336b-49e4-b165-0b9fe73cb526-istio-data\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn\" (UID: \"a9d90975-336b-49e4-b165-0b9fe73cb526\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn"
Oct 30 22:51:04.405294 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.404733    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9d90975-336b-49e4-b165-0b9fe73cb526-credential-socket\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn\" (UID: \"a9d90975-336b-49e4-b165-0b9fe73cb526\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn"
Oct 30 22:51:04.405294 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.405222    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9d90975-336b-49e4-b165-0b9fe73cb526-istiod-ca-cert\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn\" (UID: \"a9d90975-336b-49e4-b165-0b9fe73cb526\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn"
Oct 30 22:51:04.407009 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.406983    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9d90975-336b-49e4-b165-0b9fe73cb526-istio-podinfo\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn\" (UID: \"a9d90975-336b-49e4-b165-0b9fe73cb526\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn"
Oct 30 22:51:04.407539 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.407519    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9d90975-336b-49e4-b165-0b9fe73cb526-istio-envoy\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn\" (UID: \"a9d90975-336b-49e4-b165-0b9fe73cb526\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn"
Oct 30 22:51:04.468394 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.468360    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9d90975-336b-49e4-b165-0b9fe73cb526-istio-token\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn\" (UID: \"a9d90975-336b-49e4-b165-0b9fe73cb526\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn"
Oct 30 22:51:04.468651 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.468628    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z87zg\" (UniqueName: \"kubernetes.io/projected/a9d90975-336b-49e4-b165-0b9fe73cb526-kube-api-access-z87zg\") pod \"e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn\" (UID: \"a9d90975-336b-49e4-b165-0b9fe73cb526\") " pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn"
Oct 30 22:51:04.581944 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.581851    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn"
Oct 30 22:51:04.721411 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.721354    4871 patch_prober.go:29] interesting pod/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:51:04.721854 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:04.721416    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x" podUID="ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:51:05.024024 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:05.023984    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:51:05.024163 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:05.024066    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:51:05.024163 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:05.024095    4871 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:51:05.027289 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:05.027264    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn"]
Oct 30 22:51:05.104366 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:05.104327    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn" event={"ID":"a9d90975-336b-49e4-b165-0b9fe73cb526","Type":"ContainerStarted","Data":"d2472180fda96ccd2f9a3677966659e593d52fc1b4673a02c25152b83c8e27bf"}
Oct 30 22:51:05.104563 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:05.104395    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn" event={"ID":"a9d90975-336b-49e4-b165-0b9fe73cb526","Type":"ContainerStarted","Data":"94ebb5072fbf83284aa4ea9537d1bb7546d65f7b3d587ad0465068ffa760b86c"}
Oct 30 22:51:05.246914 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:05.246857    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn" podStartSLOduration=1.246839383 podStartE2EDuration="1.246839383s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:51:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:51:05.231867996 +0000 UTC m=+4407.409356612" watchObservedRunningTime="2026-10-30 22:51:05.246839383 +0000 UTC m=+4407.424327999"
Oct 30 22:51:05.582469 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:05.582414    4871 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn"
Oct 30 22:51:05.587247 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:51:05.587196    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:51:06.583641 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:06.583603    4871 patch_prober.go:29] interesting pod/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.203:15021/healthz/ready\": dial tcp 10.133.0.203:15021: connect: connection refused" start-of-body=
Oct 30 22:51:06.584053 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:06.583662    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn" podUID="a9d90975-336b-49e4-b165-0b9fe73cb526" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.203:15021/healthz/ready\": dial tcp 10.133.0.203:15021: connect: connection refused"
Oct 30 22:51:06.696264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:06.693756    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6"]
Oct 30 22:51:06.696264 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:06.694061    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6" podUID="b461178b-38cb-446a-8b6c-3a615c77a6f5" containerName="maas-api" containerID="cri-o://16c39c700557679bf8574fb9d7e2b1ac2c19883f0f18e5bd28c24509566599dc" gracePeriod=30
Oct 30 22:51:06.782119 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:06.782084    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:07.059089 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:07.059053    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6"
Oct 30 22:51:07.152697 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:07.152662    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b461178b-38cb-446a-8b6c-3a615c77a6f5-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b461178b-38cb-446a-8b6c-3a615c77a6f5-maas-api-tls\") pod \"b461178b-38cb-446a-8b6c-3a615c77a6f5\" (UID: \"b461178b-38cb-446a-8b6c-3a615c77a6f5\") "
Oct 30 22:51:07.152912 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:07.152733    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b461178b-38cb-446a-8b6c-3a615c77a6f5-kube-api-access-9px97\" (UniqueName: \"kubernetes.io/projected/b461178b-38cb-446a-8b6c-3a615c77a6f5-kube-api-access-9px97\") pod \"b461178b-38cb-446a-8b6c-3a615c77a6f5\" (UID: \"b461178b-38cb-446a-8b6c-3a615c77a6f5\") "
Oct 30 22:51:07.152912 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:07.152901    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b461178b-38cb-446a-8b6c-3a615c77a6f5-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b461178b-38cb-446a-8b6c-3a615c77a6f5-metrics-certs\") pod \"b461178b-38cb-446a-8b6c-3a615c77a6f5\" (UID: \"b461178b-38cb-446a-8b6c-3a615c77a6f5\") "
Oct 30 22:51:07.153279 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:07.153249    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b461178b-38cb-446a-8b6c-3a615c77a6f5-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b461178b-38cb-446a-8b6c-3a615c77a6f5-openshift-service-ca\") pod \"b461178b-38cb-446a-8b6c-3a615c77a6f5\" (UID: \"b461178b-38cb-446a-8b6c-3a615c77a6f5\") "
Oct 30 22:51:07.153613 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:07.153588    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b461178b-38cb-446a-8b6c-3a615c77a6f5-openshift-service-ca" pod "b461178b-38cb-446a-8b6c-3a615c77a6f5" (UID: "b461178b-38cb-446a-8b6c-3a615c77a6f5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:51:07.153981 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:07.153960    4871 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b461178b-38cb-446a-8b6c-3a615c77a6f5-openshift-service-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:07.155098 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:07.155068    4871 generic.go:356] "Generic (PLEG): container finished" podID="b461178b-38cb-446a-8b6c-3a615c77a6f5" containerID="16c39c700557679bf8574fb9d7e2b1ac2c19883f0f18e5bd28c24509566599dc" exitCode=0
Oct 30 22:51:07.155638 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:07.155378    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6" event={"ID":"b461178b-38cb-446a-8b6c-3a615c77a6f5","Type":"ContainerDied","Data":"16c39c700557679bf8574fb9d7e2b1ac2c19883f0f18e5bd28c24509566599dc"}
Oct 30 22:51:07.155638 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:07.155460    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6" event={"ID":"b461178b-38cb-446a-8b6c-3a615c77a6f5","Type":"ContainerDied","Data":"8086f82244f667d89719780092571d19d52d7f2c026e451739fc622b356f5182"}
Oct 30 22:51:07.155638 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:07.155492    4871 scope.go:122] "RemoveContainer" containerID="16c39c700557679bf8574fb9d7e2b1ac2c19883f0f18e5bd28c24509566599dc"
Oct 30 22:51:07.155638 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:07.155571    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b461178b-38cb-446a-8b6c-3a615c77a6f5-kube-api-access-9px97" pod "b461178b-38cb-446a-8b6c-3a615c77a6f5" (UID: "b461178b-38cb-446a-8b6c-3a615c77a6f5"). InnerVolumeSpecName "kube-api-access-9px97". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:51:07.155935 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:07.155733    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6"
Oct 30 22:51:07.156472 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:07.156446    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b461178b-38cb-446a-8b6c-3a615c77a6f5-metrics-certs" pod "b461178b-38cb-446a-8b6c-3a615c77a6f5" (UID: "b461178b-38cb-446a-8b6c-3a615c77a6f5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:51:07.160510 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:07.160360    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b461178b-38cb-446a-8b6c-3a615c77a6f5-maas-api-tls" pod "b461178b-38cb-446a-8b6c-3a615c77a6f5" (UID: "b461178b-38cb-446a-8b6c-3a615c77a6f5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:51:07.188028 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:07.187995    4871 scope.go:122] "RemoveContainer" containerID="16c39c700557679bf8574fb9d7e2b1ac2c19883f0f18e5bd28c24509566599dc"
Oct 30 22:51:07.188813 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:51:07.188417    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16c39c700557679bf8574fb9d7e2b1ac2c19883f0f18e5bd28c24509566599dc\": container with ID starting with 16c39c700557679bf8574fb9d7e2b1ac2c19883f0f18e5bd28c24509566599dc not found: ID does not exist" containerID="16c39c700557679bf8574fb9d7e2b1ac2c19883f0f18e5bd28c24509566599dc"
Oct 30 22:51:07.188813 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:07.188453    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16c39c700557679bf8574fb9d7e2b1ac2c19883f0f18e5bd28c24509566599dc"} err="failed to get container status \"16c39c700557679bf8574fb9d7e2b1ac2c19883f0f18e5bd28c24509566599dc\": rpc error: code = NotFound desc = could not find container \"16c39c700557679bf8574fb9d7e2b1ac2c19883f0f18e5bd28c24509566599dc\": container with ID starting with 16c39c700557679bf8574fb9d7e2b1ac2c19883f0f18e5bd28c24509566599dc not found: ID does not exist"
Oct 30 22:51:07.254574 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:07.254532    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b461178b-38cb-446a-8b6c-3a615c77a6f5-maas-api-tls\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:07.254574 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:07.254563    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9px97\" (UniqueName: \"kubernetes.io/projected/b461178b-38cb-446a-8b6c-3a615c77a6f5-kube-api-access-9px97\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:07.254574 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:07.254579    4871 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b461178b-38cb-446a-8b6c-3a615c77a6f5-metrics-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:07.583018 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:07.582980    4871 patch_prober.go:29] interesting pod/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.203:15021/healthz/ready\": dial tcp 10.133.0.203:15021: connect: connection refused" start-of-body=
Oct 30 22:51:07.583269 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:07.583044    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn" podUID="a9d90975-336b-49e4-b165-0b9fe73cb526" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.203:15021/healthz/ready\": dial tcp 10.133.0.203:15021: connect: connection refused"
Oct 30 22:51:07.588441 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:07.588409    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6"]
Oct 30 22:51:07.593439 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:07.593406    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-43d592-fd67bcf8-b94h6"]
Oct 30 22:51:07.961315 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:07.961270    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b461178b-38cb-446a-8b6c-3a615c77a6f5" path="/var/lib/kubelet/pods/b461178b-38cb-446a-8b6c-3a615c77a6f5/volumes"
Oct 30 22:51:08.453944 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:08.453914    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:08.582899 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:08.582867    4871 patch_prober.go:29] interesting pod/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.203:15021/healthz/ready\": dial tcp 10.133.0.203:15021: connect: connection refused" start-of-body=
Oct 30 22:51:08.583082 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:08.582920    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn" podUID="a9d90975-336b-49e4-b165-0b9fe73cb526" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.203:15021/healthz/ready\": dial tcp 10.133.0.203:15021: connect: connection refused"
Oct 30 22:51:08.620798 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:08.620718    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:08.781971 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:08.781885    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:09.172577 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:09.172545    4871 generic.go:356] "Generic (PLEG): container finished" podID="ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea" containerID="a7d1b0220ae04364eacc4053d39b06b2ef39977da2a77ac4708ec91e40e2a431" exitCode=0
Oct 30 22:51:09.172745 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:09.172598    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x" event={"ID":"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea","Type":"ContainerDied","Data":"a7d1b0220ae04364eacc4053d39b06b2ef39977da2a77ac4708ec91e40e2a431"}
Oct 30 22:51:09.172745 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:09.172645    4871 scope.go:122] "RemoveContainer" containerID="b5e0536939bf1528ad23a40a01e4190d2a8a8b88d444591654dd83060b68595a"
Oct 30 22:51:09.300662 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:09.300632    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"
Oct 30 22:51:09.366890 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:09.366854    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:09.379072 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:09.379040    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-istiod-ca-cert\") pod \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\" (UID: \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\") "
Oct 30 22:51:09.379227 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:09.379092    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-kube-api-access-zd8vg\" (UniqueName: \"kubernetes.io/projected/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-kube-api-access-zd8vg\") pod \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\" (UID: \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\") "
Oct 30 22:51:09.379227 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:09.379139    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-workload-certs\") pod \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\" (UID: \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\") "
Oct 30 22:51:09.379227 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:09.379162    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-istio-token\" (UniqueName: \"kubernetes.io/projected/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-istio-token\") pod \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\" (UID: \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\") "
Oct 30 22:51:09.379227 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:09.379196    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-istio-podinfo\") pod \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\" (UID: \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\") "
Oct 30 22:51:09.379463 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:09.379276    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-workload-socket\") pod \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\" (UID: \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\") "
Oct 30 22:51:09.379463 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:09.379386    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-istio-envoy\") pod \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\" (UID: \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\") "
Oct 30 22:51:09.379463 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:09.379405    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-istio-data\") pod \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\" (UID: \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\") "
Oct 30 22:51:09.379463 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:09.379428    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-credential-socket\") pod \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\" (UID: \"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea\") "
Oct 30 22:51:09.379689 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:09.379536    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-workload-certs" pod "ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea" (UID: "ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:51:09.379689 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:09.379587    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-istiod-ca-cert" pod "ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea" (UID: "ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:51:09.379811 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:09.379743    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-istio-data" pod "ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea" (UID: "ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:51:09.379998 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:09.379905    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-workload-socket" pod "ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea" (UID: "ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:51:09.379998 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:09.379920    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-credential-socket" pod "ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea" (UID: "ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:51:09.379998 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:09.379977    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:09.379998 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:09.379994    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:09.380208 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:09.380010    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:09.381629 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:09.381608    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-istio-token" pod "ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea" (UID: "ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:51:09.381717 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:09.381633    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-istio-envoy" pod "ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea" (UID: "ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:51:09.381929 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:09.381912    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-istio-podinfo" pod "ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea" (UID: "ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:51:09.381971 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:09.381933    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-kube-api-access-zd8vg" pod "ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea" (UID: "ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea"). InnerVolumeSpecName "kube-api-access-zd8vg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:51:09.480963 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:09.480923    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zd8vg\" (UniqueName: \"kubernetes.io/projected/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-kube-api-access-zd8vg\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:09.480963 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:09.480958    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:09.480963 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:09.480973    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:09.481229 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:09.480986    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:09.481229 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:09.481001    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:09.481229 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:09.481025    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:09.582246 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:09.582196    4871 patch_prober.go:29] interesting pod/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.203:15021/healthz/ready\": dial tcp 10.133.0.203:15021: connect: connection refused" start-of-body=
Oct 30 22:51:09.582417 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:09.582268    4871 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn" podUID="a9d90975-336b-49e4-b165-0b9fe73cb526" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.203:15021/healthz/ready\": dial tcp 10.133.0.203:15021: connect: connection refused"
Oct 30 22:51:10.181686 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:10.181656    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"
Oct 30 22:51:10.182162 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:10.181694    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x" event={"ID":"ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea","Type":"ContainerDied","Data":"e2328916e36c247a3268e19d96012f1f17893717728fa6e2c6d5d295f90211f4"}
Oct 30 22:51:10.182162 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:10.181762    4871 scope.go:122] "RemoveContainer" containerID="a7d1b0220ae04364eacc4053d39b06b2ef39977da2a77ac4708ec91e40e2a431"
Oct 30 22:51:10.227999 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:10.227890    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:10.416013 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:10.415937    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:10.428714 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:10.428668    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"]
Oct 30 22:51:10.464318 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:10.464277    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-5e884c-openshift-default-5b859fffbf-cgr9x"]
Oct 30 22:51:10.533465 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:10.533369    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn"]
Oct 30 22:51:10.533688 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:10.533656    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn" podUID="a9d90975-336b-49e4-b165-0b9fe73cb526" containerName="istio-proxy" containerID="cri-o://d2472180fda96ccd2f9a3677966659e593d52fc1b4673a02c25152b83c8e27bf" gracePeriod=30
Oct 30 22:51:10.567060 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:10.567024    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:11.961030 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:11.960984    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea" path="/var/lib/kubelet/pods/ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea/volumes"
Oct 30 22:51:13.371558 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:13.371523    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf"]
Oct 30 22:51:13.372277 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:13.372227    4871 cpu_manager.go:414] "containerMap: removing container" podUID="38bc129a-15c0-4019-b32d-e4b62e79b513" containerName="istio-proxy"
Oct 30 22:51:13.372277 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:13.372269    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="38bc129a-15c0-4019-b32d-e4b62e79b513" containerName="istio-proxy"
Oct 30 22:51:13.372451 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:13.372304    4871 cpu_manager.go:414] "containerMap: removing container" podUID="b461178b-38cb-446a-8b6c-3a615c77a6f5" containerName="maas-api"
Oct 30 22:51:13.372451 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:13.372312    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b461178b-38cb-446a-8b6c-3a615c77a6f5" containerName="maas-api"
Oct 30 22:51:13.372451 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:13.372337    4871 cpu_manager.go:414] "containerMap: removing container" podUID="ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea" containerName="istio-proxy"
Oct 30 22:51:13.372451 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:13.372345    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea" containerName="istio-proxy"
Oct 30 22:51:13.372451 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:13.372377    4871 cpu_manager.go:414] "containerMap: removing container" podUID="f3798606-a072-44a5-891c-4588de3c6ff1" containerName="istio-proxy"
Oct 30 22:51:13.372451 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:13.372384    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f3798606-a072-44a5-891c-4588de3c6ff1" containerName="istio-proxy"
Oct 30 22:51:13.372763 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:13.372509    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="b873ef79-e292-4496-a9ec-8d2ec09f65f6" containerName="istio-proxy"
Oct 30 22:51:13.372763 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:13.372528    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea" containerName="istio-proxy"
Oct 30 22:51:13.372763 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:13.372540    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea" containerName="istio-proxy"
Oct 30 22:51:13.372763 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:13.372551    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="f3798606-a072-44a5-891c-4588de3c6ff1" containerName="istio-proxy"
Oct 30 22:51:13.372763 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:13.372562    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="38bc129a-15c0-4019-b32d-e4b62e79b513" containerName="istio-proxy"
Oct 30 22:51:13.372763 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:13.372575    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="b461178b-38cb-446a-8b6c-3a615c77a6f5" containerName="maas-api"
Oct 30 22:51:13.372763 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:13.372586    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="6cbecab8-bfad-40bf-850a-905e4d365d29" containerName="istio-proxy"
Oct 30 22:51:13.377416 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:13.377396    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf"
Oct 30 22:51:13.433231 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:13.433193    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf"]
Oct 30 22:51:13.481614 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:13.481573    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-b-9af9e7-serving-cert\""
Oct 30 22:51:13.539570 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:13.539534    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb586958-b1df-4396-8ca4-f2bbf71addbc-metrics-certs\") pod \"maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf\" (UID: \"bb586958-b1df-4396-8ca4-f2bbf71addbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf"
Oct 30 22:51:13.539809 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:13.539650    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb586958-b1df-4396-8ca4-f2bbf71addbc-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf\" (UID: \"bb586958-b1df-4396-8ca4-f2bbf71addbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf"
Oct 30 22:51:13.539809 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:13.539681    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb586958-b1df-4396-8ca4-f2bbf71addbc-maas-api-tls\") pod \"maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf\" (UID: \"bb586958-b1df-4396-8ca4-f2bbf71addbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf"
Oct 30 22:51:13.539809 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:13.539780    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvhwz\" (UniqueName: \"kubernetes.io/projected/bb586958-b1df-4396-8ca4-f2bbf71addbc-kube-api-access-vvhwz\") pod \"maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf\" (UID: \"bb586958-b1df-4396-8ca4-f2bbf71addbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf"
Oct 30 22:51:13.557320 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:13.557288    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:13.643257 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:13.643149    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb586958-b1df-4396-8ca4-f2bbf71addbc-metrics-certs\") pod \"maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf\" (UID: \"bb586958-b1df-4396-8ca4-f2bbf71addbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf"
Oct 30 22:51:13.643613 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:13.643565    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb586958-b1df-4396-8ca4-f2bbf71addbc-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf\" (UID: \"bb586958-b1df-4396-8ca4-f2bbf71addbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf"
Oct 30 22:51:13.643756 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:13.643634    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb586958-b1df-4396-8ca4-f2bbf71addbc-maas-api-tls\") pod \"maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf\" (UID: \"bb586958-b1df-4396-8ca4-f2bbf71addbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf"
Oct 30 22:51:13.643756 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:13.643691    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvhwz\" (UniqueName: \"kubernetes.io/projected/bb586958-b1df-4396-8ca4-f2bbf71addbc-kube-api-access-vvhwz\") pod \"maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf\" (UID: \"bb586958-b1df-4396-8ca4-f2bbf71addbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf"
Oct 30 22:51:13.645074 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:51:13.645047    4871 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-b-9af9e7-serving-cert: secret "maas-api-e2e-extmod-b-9af9e7-serving-cert" not found
Oct 30 22:51:13.645208 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:51:13.645186    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb586958-b1df-4396-8ca4-f2bbf71addbc-maas-api-tls podName:bb586958-b1df-4396-8ca4-f2bbf71addbc nodeName:}" failed. No retries permitted until 2026-10-30 22:51:14.145162739 +0000 UTC m=+4416.322651353 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bb586958-b1df-4396-8ca4-f2bbf71addbc-maas-api-tls") pod "maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf" (UID: "bb586958-b1df-4396-8ca4-f2bbf71addbc") : secret "maas-api-e2e-extmod-b-9af9e7-serving-cert" not found
Oct 30 22:51:13.645395 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:13.645366    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb586958-b1df-4396-8ca4-f2bbf71addbc-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf\" (UID: \"bb586958-b1df-4396-8ca4-f2bbf71addbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf"
Oct 30 22:51:13.650490 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:13.650465    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb586958-b1df-4396-8ca4-f2bbf71addbc-metrics-certs\") pod \"maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf\" (UID: \"bb586958-b1df-4396-8ca4-f2bbf71addbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf"
Oct 30 22:51:13.708530 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:13.708489    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvhwz\" (UniqueName: \"kubernetes.io/projected/bb586958-b1df-4396-8ca4-f2bbf71addbc-kube-api-access-vvhwz\") pod \"maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf\" (UID: \"bb586958-b1df-4396-8ca4-f2bbf71addbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf"
Oct 30 22:51:13.781155 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:13.781117    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:13.932438 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:13.932343    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:14.150481 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:14.150435    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb586958-b1df-4396-8ca4-f2bbf71addbc-maas-api-tls\") pod \"maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf\" (UID: \"bb586958-b1df-4396-8ca4-f2bbf71addbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf"
Oct 30 22:51:14.153073 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:14.153045    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb586958-b1df-4396-8ca4-f2bbf71addbc-maas-api-tls\") pod \"maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf\" (UID: \"bb586958-b1df-4396-8ca4-f2bbf71addbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf"
Oct 30 22:51:14.291988 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:14.291900    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf"
Oct 30 22:51:14.673196 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:51:14.673151    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb586958_b1df_4396_8ca4_f2bbf71addbc.slice/crio-08c9afe2ca71df65dd7eabfb1d8934b20a694bb6914a590768fa2eed03ca3b8c WatchSource:0}: Error finding container 08c9afe2ca71df65dd7eabfb1d8934b20a694bb6914a590768fa2eed03ca3b8c: Status 404 returned error can't find the container with id 08c9afe2ca71df65dd7eabfb1d8934b20a694bb6914a590768fa2eed03ca3b8c
Oct 30 22:51:14.675033 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:14.675010    4871 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:51:14.677067 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:14.677040    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf"]
Oct 30 22:51:15.220302 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:15.220233    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf" event={"ID":"bb586958-b1df-4396-8ca4-f2bbf71addbc","Type":"ContainerStarted","Data":"08c9afe2ca71df65dd7eabfb1d8934b20a694bb6914a590768fa2eed03ca3b8c"}
Oct 30 22:51:15.525744 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:15.525665    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:15.786933 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:15.786846    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:15.885088 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:15.885050    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"]
Oct 30 22:51:15.885366 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:15.885336    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s" podUID="239ef6a2-b9f8-4247-9db4-fc4192f6bf1c" containerName="istio-proxy" containerID="cri-o://7e7a69db017786d9da7baa4acc750763a8155a6f38da5542554caa52b33c2157" gracePeriod=30
Oct 30 22:51:15.970202 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:15.970170    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:16.229066 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:16.229026    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf" event={"ID":"bb586958-b1df-4396-8ca4-f2bbf71addbc","Type":"ContainerStarted","Data":"4d08f2354fbca3486078415b96e42d8f4305db0c998c0a934500a6bfe8e3b3b6"}
Oct 30 22:51:16.229283 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:16.229157    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf"
Oct 30 22:51:16.367809 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:16.367729    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf" podStartSLOduration=2.493673932 podStartE2EDuration="3.367710958s" totalImagesPullingTime="874.037026ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:51:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:51:16.365669875 +0000 UTC m=+4418.543158493" watchObservedRunningTime="2026-10-30 22:51:16.367710958 +0000 UTC m=+4418.545199576"
Oct 30 22:51:17.810774 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:17.810744    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn"
Oct 30 22:51:17.907974 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:17.907938    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9d90975-336b-49e4-b165-0b9fe73cb526-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9d90975-336b-49e4-b165-0b9fe73cb526-workload-certs\") pod \"a9d90975-336b-49e4-b165-0b9fe73cb526\" (UID: \"a9d90975-336b-49e4-b165-0b9fe73cb526\") "
Oct 30 22:51:17.908141 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:17.908029    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a9d90975-336b-49e4-b165-0b9fe73cb526-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9d90975-336b-49e4-b165-0b9fe73cb526-istiod-ca-cert\") pod \"a9d90975-336b-49e4-b165-0b9fe73cb526\" (UID: \"a9d90975-336b-49e4-b165-0b9fe73cb526\") "
Oct 30 22:51:17.908141 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:17.908060    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9d90975-336b-49e4-b165-0b9fe73cb526-istio-token\" (UniqueName: \"kubernetes.io/projected/a9d90975-336b-49e4-b165-0b9fe73cb526-istio-token\") pod \"a9d90975-336b-49e4-b165-0b9fe73cb526\" (UID: \"a9d90975-336b-49e4-b165-0b9fe73cb526\") "
Oct 30 22:51:17.908141 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:17.908102    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9d90975-336b-49e4-b165-0b9fe73cb526-kube-api-access-z87zg\" (UniqueName: \"kubernetes.io/projected/a9d90975-336b-49e4-b165-0b9fe73cb526-kube-api-access-z87zg\") pod \"a9d90975-336b-49e4-b165-0b9fe73cb526\" (UID: \"a9d90975-336b-49e4-b165-0b9fe73cb526\") "
Oct 30 22:51:17.908141 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:17.908126    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9d90975-336b-49e4-b165-0b9fe73cb526-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9d90975-336b-49e4-b165-0b9fe73cb526-credential-socket\") pod \"a9d90975-336b-49e4-b165-0b9fe73cb526\" (UID: \"a9d90975-336b-49e4-b165-0b9fe73cb526\") "
Oct 30 22:51:17.908429 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:17.908164    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9d90975-336b-49e4-b165-0b9fe73cb526-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9d90975-336b-49e4-b165-0b9fe73cb526-istio-envoy\") pod \"a9d90975-336b-49e4-b165-0b9fe73cb526\" (UID: \"a9d90975-336b-49e4-b165-0b9fe73cb526\") "
Oct 30 22:51:17.908429 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:17.908331    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a9d90975-336b-49e4-b165-0b9fe73cb526-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9d90975-336b-49e4-b165-0b9fe73cb526-istio-podinfo\") pod \"a9d90975-336b-49e4-b165-0b9fe73cb526\" (UID: \"a9d90975-336b-49e4-b165-0b9fe73cb526\") "
Oct 30 22:51:17.908429 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:17.908397    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9d90975-336b-49e4-b165-0b9fe73cb526-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9d90975-336b-49e4-b165-0b9fe73cb526-istio-data\") pod \"a9d90975-336b-49e4-b165-0b9fe73cb526\" (UID: \"a9d90975-336b-49e4-b165-0b9fe73cb526\") "
Oct 30 22:51:17.908588 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:17.908557    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9d90975-336b-49e4-b165-0b9fe73cb526-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9d90975-336b-49e4-b165-0b9fe73cb526-workload-socket\") pod \"a9d90975-336b-49e4-b165-0b9fe73cb526\" (UID: \"a9d90975-336b-49e4-b165-0b9fe73cb526\") "
Oct 30 22:51:17.908588 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:17.908335    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9d90975-336b-49e4-b165-0b9fe73cb526-workload-certs" pod "a9d90975-336b-49e4-b165-0b9fe73cb526" (UID: "a9d90975-336b-49e4-b165-0b9fe73cb526"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:51:17.908588 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:17.908556    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9d90975-336b-49e4-b165-0b9fe73cb526-istiod-ca-cert" pod "a9d90975-336b-49e4-b165-0b9fe73cb526" (UID: "a9d90975-336b-49e4-b165-0b9fe73cb526"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:51:17.908588 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:17.908588    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9d90975-336b-49e4-b165-0b9fe73cb526-credential-socket" pod "a9d90975-336b-49e4-b165-0b9fe73cb526" (UID: "a9d90975-336b-49e4-b165-0b9fe73cb526"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:51:17.908871 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:17.908765    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9d90975-336b-49e4-b165-0b9fe73cb526-istio-data" pod "a9d90975-336b-49e4-b165-0b9fe73cb526" (UID: "a9d90975-336b-49e4-b165-0b9fe73cb526"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:51:17.908936 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:17.908873    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9d90975-336b-49e4-b165-0b9fe73cb526-workload-socket" pod "a9d90975-336b-49e4-b165-0b9fe73cb526" (UID: "a9d90975-336b-49e4-b165-0b9fe73cb526"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:51:17.909139 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:17.909116    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9d90975-336b-49e4-b165-0b9fe73cb526-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:17.909210 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:17.909138    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9d90975-336b-49e4-b165-0b9fe73cb526-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:17.909210 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:17.909153    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9d90975-336b-49e4-b165-0b9fe73cb526-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:17.909210 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:17.909167    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9d90975-336b-49e4-b165-0b9fe73cb526-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:17.909210 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:17.909183    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9d90975-336b-49e4-b165-0b9fe73cb526-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:17.911085 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:17.911049    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a9d90975-336b-49e4-b165-0b9fe73cb526-istio-podinfo" pod "a9d90975-336b-49e4-b165-0b9fe73cb526" (UID: "a9d90975-336b-49e4-b165-0b9fe73cb526"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:51:17.911195 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:17.911091    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9d90975-336b-49e4-b165-0b9fe73cb526-istio-envoy" pod "a9d90975-336b-49e4-b165-0b9fe73cb526" (UID: "a9d90975-336b-49e4-b165-0b9fe73cb526"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:51:17.911195 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:17.911093    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9d90975-336b-49e4-b165-0b9fe73cb526-kube-api-access-z87zg" pod "a9d90975-336b-49e4-b165-0b9fe73cb526" (UID: "a9d90975-336b-49e4-b165-0b9fe73cb526"). InnerVolumeSpecName "kube-api-access-z87zg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:51:17.911195 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:17.911121    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9d90975-336b-49e4-b165-0b9fe73cb526-istio-token" pod "a9d90975-336b-49e4-b165-0b9fe73cb526" (UID: "a9d90975-336b-49e4-b165-0b9fe73cb526"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:51:18.010683 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:18.010637    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9d90975-336b-49e4-b165-0b9fe73cb526-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:18.010882 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:18.010694    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z87zg\" (UniqueName: \"kubernetes.io/projected/a9d90975-336b-49e4-b165-0b9fe73cb526-kube-api-access-z87zg\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:18.010882 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:18.010712    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9d90975-336b-49e4-b165-0b9fe73cb526-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:18.010882 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:18.010727    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9d90975-336b-49e4-b165-0b9fe73cb526-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:18.245270 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:18.245162    4871 generic.go:356] "Generic (PLEG): container finished" podID="a9d90975-336b-49e4-b165-0b9fe73cb526" containerID="d2472180fda96ccd2f9a3677966659e593d52fc1b4673a02c25152b83c8e27bf" exitCode=0
Oct 30 22:51:18.245270 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:18.245214    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn"
Oct 30 22:51:18.245270 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:18.245256    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn" event={"ID":"a9d90975-336b-49e4-b165-0b9fe73cb526","Type":"ContainerDied","Data":"d2472180fda96ccd2f9a3677966659e593d52fc1b4673a02c25152b83c8e27bf"}
Oct 30 22:51:18.245587 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:18.245318    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn" event={"ID":"a9d90975-336b-49e4-b165-0b9fe73cb526","Type":"ContainerDied","Data":"94ebb5072fbf83284aa4ea9537d1bb7546d65f7b3d587ad0465068ffa760b86c"}
Oct 30 22:51:18.245587 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:18.245342    4871 scope.go:122] "RemoveContainer" containerID="d2472180fda96ccd2f9a3677966659e593d52fc1b4673a02c25152b83c8e27bf"
Oct 30 22:51:18.257002 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:18.256979    4871 scope.go:122] "RemoveContainer" containerID="d2472180fda96ccd2f9a3677966659e593d52fc1b4673a02c25152b83c8e27bf"
Oct 30 22:51:18.257363 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:51:18.257342    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2472180fda96ccd2f9a3677966659e593d52fc1b4673a02c25152b83c8e27bf\": container with ID starting with d2472180fda96ccd2f9a3677966659e593d52fc1b4673a02c25152b83c8e27bf not found: ID does not exist" containerID="d2472180fda96ccd2f9a3677966659e593d52fc1b4673a02c25152b83c8e27bf"
Oct 30 22:51:18.257434 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:18.257371    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2472180fda96ccd2f9a3677966659e593d52fc1b4673a02c25152b83c8e27bf"} err="failed to get container status \"d2472180fda96ccd2f9a3677966659e593d52fc1b4673a02c25152b83c8e27bf\": rpc error: code = NotFound desc = could not find container \"d2472180fda96ccd2f9a3677966659e593d52fc1b4673a02c25152b83c8e27bf\": container with ID starting with d2472180fda96ccd2f9a3677966659e593d52fc1b4673a02c25152b83c8e27bf not found: ID does not exist"
Oct 30 22:51:18.419373 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:18.419335    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn"]
Oct 30 22:51:18.422563 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:18.422530    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-26743a-openshift-default-56b6f9f987-qxxcn"]
Oct 30 22:51:18.702507 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:18.702467    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:19.959794 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:19.959756    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a9d90975-336b-49e4-b165-0b9fe73cb526" path="/var/lib/kubelet/pods/a9d90975-336b-49e4-b165-0b9fe73cb526/volumes"
Oct 30 22:51:21.163060 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.162986    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"
Oct 30 22:51:21.249473 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.249433    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-istio-envoy\") pod \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\" (UID: \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\") "
Oct 30 22:51:21.249691 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.249538    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-istio-data\") pod \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\" (UID: \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\") "
Oct 30 22:51:21.249691 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.249600    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-kube-api-access-jvztn\" (UniqueName: \"kubernetes.io/projected/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-kube-api-access-jvztn\") pod \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\" (UID: \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\") "
Oct 30 22:51:21.249691 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.249619    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-workload-socket\") pod \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\" (UID: \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\") "
Oct 30 22:51:21.249691 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.249666    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-istio-podinfo\") pod \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\" (UID: \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\") "
Oct 30 22:51:21.249691 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.249682    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-istio-token\" (UniqueName: \"kubernetes.io/projected/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-istio-token\") pod \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\" (UID: \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\") "
Oct 30 22:51:21.249995 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.249724    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-credential-socket\") pod \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\" (UID: \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\") "
Oct 30 22:51:21.249995 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.249767    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-workload-certs\") pod \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\" (UID: \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\") "
Oct 30 22:51:21.249995 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.249814    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-istiod-ca-cert\") pod \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\" (UID: \"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c\") "
Oct 30 22:51:21.249995 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.249860    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-istio-data" pod "239ef6a2-b9f8-4247-9db4-fc4192f6bf1c" (UID: "239ef6a2-b9f8-4247-9db4-fc4192f6bf1c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:51:21.249995 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.249923    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-workload-socket" pod "239ef6a2-b9f8-4247-9db4-fc4192f6bf1c" (UID: "239ef6a2-b9f8-4247-9db4-fc4192f6bf1c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:51:21.250288 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.250150    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-credential-socket" pod "239ef6a2-b9f8-4247-9db4-fc4192f6bf1c" (UID: "239ef6a2-b9f8-4247-9db4-fc4192f6bf1c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:51:21.250288 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.250210    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-workload-certs" pod "239ef6a2-b9f8-4247-9db4-fc4192f6bf1c" (UID: "239ef6a2-b9f8-4247-9db4-fc4192f6bf1c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:51:21.250409 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.250292    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-istiod-ca-cert" pod "239ef6a2-b9f8-4247-9db4-fc4192f6bf1c" (UID: "239ef6a2-b9f8-4247-9db4-fc4192f6bf1c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:51:21.250409 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.250338    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:21.250409 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.250358    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:21.250409 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.250378    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:21.251903 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.251868    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-istio-envoy" pod "239ef6a2-b9f8-4247-9db4-fc4192f6bf1c" (UID: "239ef6a2-b9f8-4247-9db4-fc4192f6bf1c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:51:21.252019 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.251938    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-istio-token" pod "239ef6a2-b9f8-4247-9db4-fc4192f6bf1c" (UID: "239ef6a2-b9f8-4247-9db4-fc4192f6bf1c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:51:21.252062 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.252028    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-kube-api-access-jvztn" pod "239ef6a2-b9f8-4247-9db4-fc4192f6bf1c" (UID: "239ef6a2-b9f8-4247-9db4-fc4192f6bf1c"). InnerVolumeSpecName "kube-api-access-jvztn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:51:21.252138 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.252118    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-istio-podinfo" pod "239ef6a2-b9f8-4247-9db4-fc4192f6bf1c" (UID: "239ef6a2-b9f8-4247-9db4-fc4192f6bf1c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:51:21.270848 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.270816    4871 generic.go:356] "Generic (PLEG): container finished" podID="239ef6a2-b9f8-4247-9db4-fc4192f6bf1c" containerID="7e7a69db017786d9da7baa4acc750763a8155a6f38da5542554caa52b33c2157" exitCode=0
Oct 30 22:51:21.271026 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.270879    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"
Oct 30 22:51:21.271026 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.270898    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s" event={"ID":"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c","Type":"ContainerDied","Data":"7e7a69db017786d9da7baa4acc750763a8155a6f38da5542554caa52b33c2157"}
Oct 30 22:51:21.271026 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.270958    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s" event={"ID":"239ef6a2-b9f8-4247-9db4-fc4192f6bf1c","Type":"ContainerDied","Data":"690386ef1dc45876f8d3a38792bf6fe64cc7b717e085082bdf037fd8c980a301"}
Oct 30 22:51:21.271026 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.270982    4871 scope.go:122] "RemoveContainer" containerID="7e7a69db017786d9da7baa4acc750763a8155a6f38da5542554caa52b33c2157"
Oct 30 22:51:21.282065 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.282045    4871 scope.go:122] "RemoveContainer" containerID="373bdb1823397318e10f3e052140772b7c13846f314a3bb9840627af7ca2c82a"
Oct 30 22:51:21.291933 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.291910    4871 scope.go:122] "RemoveContainer" containerID="7e7a69db017786d9da7baa4acc750763a8155a6f38da5542554caa52b33c2157"
Oct 30 22:51:21.292225 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:51:21.292208    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e7a69db017786d9da7baa4acc750763a8155a6f38da5542554caa52b33c2157\": container with ID starting with 7e7a69db017786d9da7baa4acc750763a8155a6f38da5542554caa52b33c2157 not found: ID does not exist" containerID="7e7a69db017786d9da7baa4acc750763a8155a6f38da5542554caa52b33c2157"
Oct 30 22:51:21.292300 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.292245    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e7a69db017786d9da7baa4acc750763a8155a6f38da5542554caa52b33c2157"} err="failed to get container status \"7e7a69db017786d9da7baa4acc750763a8155a6f38da5542554caa52b33c2157\": rpc error: code = NotFound desc = could not find container \"7e7a69db017786d9da7baa4acc750763a8155a6f38da5542554caa52b33c2157\": container with ID starting with 7e7a69db017786d9da7baa4acc750763a8155a6f38da5542554caa52b33c2157 not found: ID does not exist"
Oct 30 22:51:21.292300 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.292264    4871 scope.go:122] "RemoveContainer" containerID="373bdb1823397318e10f3e052140772b7c13846f314a3bb9840627af7ca2c82a"
Oct 30 22:51:21.292554 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:51:21.292528    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"373bdb1823397318e10f3e052140772b7c13846f314a3bb9840627af7ca2c82a\": container with ID starting with 373bdb1823397318e10f3e052140772b7c13846f314a3bb9840627af7ca2c82a not found: ID does not exist" containerID="373bdb1823397318e10f3e052140772b7c13846f314a3bb9840627af7ca2c82a"
Oct 30 22:51:21.292658 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.292558    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"373bdb1823397318e10f3e052140772b7c13846f314a3bb9840627af7ca2c82a"} err="failed to get container status \"373bdb1823397318e10f3e052140772b7c13846f314a3bb9840627af7ca2c82a\": rpc error: code = NotFound desc = could not find container \"373bdb1823397318e10f3e052140772b7c13846f314a3bb9840627af7ca2c82a\": container with ID starting with 373bdb1823397318e10f3e052140772b7c13846f314a3bb9840627af7ca2c82a not found: ID does not exist"
Oct 30 22:51:21.351892 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.351851    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:21.351892 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.351888    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:21.352108 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.351902    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:21.352108 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.351911    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvztn\" (UniqueName: \"kubernetes.io/projected/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-kube-api-access-jvztn\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:21.352108 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.351924    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:21.352108 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.351934    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:21.464034 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.463936    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"]
Oct 30 22:51:21.471805 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.471768    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-43d592-openshift-default-89674766c-zwx9s"]
Oct 30 22:51:21.961715 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:21.961672    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="239ef6a2-b9f8-4247-9db4-fc4192f6bf1c" path="/var/lib/kubelet/pods/239ef6a2-b9f8-4247-9db4-fc4192f6bf1c/volumes"
Oct 30 22:51:22.244140 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:22.244050    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf"
Oct 30 22:51:23.505423 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:23.505383    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:27.492206 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:27.492160    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:27.986469 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:27.986438    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:28.143400 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:28.143365    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:29.209456 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:29.209415    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:32.443684 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:32.443649    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:32.694881 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:32.694798    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:33.403968 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:33.403927    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:33.996131 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:33.996091    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:35.568502 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:51:35.568467    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:51:38.702769 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:38.702735    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:39.411990 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:39.411930    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:39.579407 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:39.579368    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:39.807737 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:39.807655    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"]
Oct 30 22:51:39.808170 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:39.807846    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh" podUID="167cb6f4-5f3b-4e79-bd45-3ffc48cefa33" containerName="istio-proxy" containerID="cri-o://c3fa0a9489e5af1ddca60646f80f479bf69721af1329409f40b84b67b8f21854" gracePeriod=30
Oct 30 22:51:39.812124 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:39.812098    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:40.297848 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:40.297813    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:43.554127 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:43.554098    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:45.078150 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.078082    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"
Oct 30 22:51:45.165333 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.165288    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-credential-socket\") pod \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\" (UID: \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\") "
Oct 30 22:51:45.165549 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.165350    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-workload-certs\") pod \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\" (UID: \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\") "
Oct 30 22:51:45.165549 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.165446    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-istio-envoy\") pod \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\" (UID: \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\") "
Oct 30 22:51:45.165549 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.165486    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-istio-data\") pod \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\" (UID: \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\") "
Oct 30 22:51:45.165549 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.165521    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-istio-token\" (UniqueName: \"kubernetes.io/projected/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-istio-token\") pod \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\" (UID: \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\") "
Oct 30 22:51:45.165549 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.165546    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-istiod-ca-cert\") pod \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\" (UID: \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\") "
Oct 30 22:51:45.165866 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.165585    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-kube-api-access-zv2qk\" (UniqueName: \"kubernetes.io/projected/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-kube-api-access-zv2qk\") pod \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\" (UID: \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\") "
Oct 30 22:51:45.165866 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.165619    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-credential-socket" pod "167cb6f4-5f3b-4e79-bd45-3ffc48cefa33" (UID: "167cb6f4-5f3b-4e79-bd45-3ffc48cefa33"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:51:45.165866 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.165642    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-workload-socket\") pod \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\" (UID: \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\") "
Oct 30 22:51:45.165866 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.165708    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-istio-podinfo\") pod \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\" (UID: \"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33\") "
Oct 30 22:51:45.165866 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.165757    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-workload-certs" pod "167cb6f4-5f3b-4e79-bd45-3ffc48cefa33" (UID: "167cb6f4-5f3b-4e79-bd45-3ffc48cefa33"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:51:45.165866 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.165806    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-istio-data" pod "167cb6f4-5f3b-4e79-bd45-3ffc48cefa33" (UID: "167cb6f4-5f3b-4e79-bd45-3ffc48cefa33"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:51:45.166211 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.165969    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-istiod-ca-cert" pod "167cb6f4-5f3b-4e79-bd45-3ffc48cefa33" (UID: "167cb6f4-5f3b-4e79-bd45-3ffc48cefa33"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:51:45.166211 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.166023    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-workload-socket" pod "167cb6f4-5f3b-4e79-bd45-3ffc48cefa33" (UID: "167cb6f4-5f3b-4e79-bd45-3ffc48cefa33"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:51:45.166211 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.166174    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:45.166211 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.166198    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:45.166475 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.166213    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:45.166475 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.166228    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:45.166475 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.166275    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:45.167991 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.167964    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-kube-api-access-zv2qk" pod "167cb6f4-5f3b-4e79-bd45-3ffc48cefa33" (UID: "167cb6f4-5f3b-4e79-bd45-3ffc48cefa33"). InnerVolumeSpecName "kube-api-access-zv2qk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:51:45.168116 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.167998    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-istio-token" pod "167cb6f4-5f3b-4e79-bd45-3ffc48cefa33" (UID: "167cb6f4-5f3b-4e79-bd45-3ffc48cefa33"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:51:45.168116 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.168050    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-istio-envoy" pod "167cb6f4-5f3b-4e79-bd45-3ffc48cefa33" (UID: "167cb6f4-5f3b-4e79-bd45-3ffc48cefa33"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:51:45.168116 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.168046    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-istio-podinfo" pod "167cb6f4-5f3b-4e79-bd45-3ffc48cefa33" (UID: "167cb6f4-5f3b-4e79-bd45-3ffc48cefa33"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:51:45.267684 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.267639    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:45.267684 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.267681    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:45.267919 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.267698    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zv2qk\" (UniqueName: \"kubernetes.io/projected/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-kube-api-access-zv2qk\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:45.267919 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.267710    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:45.409610 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.409576    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:45.469904 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.469871    4871 generic.go:356] "Generic (PLEG): container finished" podID="167cb6f4-5f3b-4e79-bd45-3ffc48cefa33" containerID="c3fa0a9489e5af1ddca60646f80f479bf69721af1329409f40b84b67b8f21854" exitCode=0
Oct 30 22:51:45.470096 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.469931    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"
Oct 30 22:51:45.470096 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.469965    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh" event={"ID":"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33","Type":"ContainerDied","Data":"c3fa0a9489e5af1ddca60646f80f479bf69721af1329409f40b84b67b8f21854"}
Oct 30 22:51:45.470096 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.470027    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh" event={"ID":"167cb6f4-5f3b-4e79-bd45-3ffc48cefa33","Type":"ContainerDied","Data":"32243cfc36fee7052139aca3db90481be0d62774b92fbc7174a6da4f72fee6d3"}
Oct 30 22:51:45.470096 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.470058    4871 scope.go:122] "RemoveContainer" containerID="c3fa0a9489e5af1ddca60646f80f479bf69721af1329409f40b84b67b8f21854"
Oct 30 22:51:45.482249 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.482216    4871 scope.go:122] "RemoveContainer" containerID="66537becb0b66142bd959161daaa772454ddfeab12f96ea2a0b0162044e1b33a"
Oct 30 22:51:45.492864 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.492841    4871 scope.go:122] "RemoveContainer" containerID="c3fa0a9489e5af1ddca60646f80f479bf69721af1329409f40b84b67b8f21854"
Oct 30 22:51:45.493218 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:51:45.493198    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3fa0a9489e5af1ddca60646f80f479bf69721af1329409f40b84b67b8f21854\": container with ID starting with c3fa0a9489e5af1ddca60646f80f479bf69721af1329409f40b84b67b8f21854 not found: ID does not exist" containerID="c3fa0a9489e5af1ddca60646f80f479bf69721af1329409f40b84b67b8f21854"
Oct 30 22:51:45.493304 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.493226    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3fa0a9489e5af1ddca60646f80f479bf69721af1329409f40b84b67b8f21854"} err="failed to get container status \"c3fa0a9489e5af1ddca60646f80f479bf69721af1329409f40b84b67b8f21854\": rpc error: code = NotFound desc = could not find container \"c3fa0a9489e5af1ddca60646f80f479bf69721af1329409f40b84b67b8f21854\": container with ID starting with c3fa0a9489e5af1ddca60646f80f479bf69721af1329409f40b84b67b8f21854 not found: ID does not exist"
Oct 30 22:51:45.493304 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.493282    4871 scope.go:122] "RemoveContainer" containerID="66537becb0b66142bd959161daaa772454ddfeab12f96ea2a0b0162044e1b33a"
Oct 30 22:51:45.493587 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:51:45.493567    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66537becb0b66142bd959161daaa772454ddfeab12f96ea2a0b0162044e1b33a\": container with ID starting with 66537becb0b66142bd959161daaa772454ddfeab12f96ea2a0b0162044e1b33a not found: ID does not exist" containerID="66537becb0b66142bd959161daaa772454ddfeab12f96ea2a0b0162044e1b33a"
Oct 30 22:51:45.493651 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.493597    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66537becb0b66142bd959161daaa772454ddfeab12f96ea2a0b0162044e1b33a"} err="failed to get container status \"66537becb0b66142bd959161daaa772454ddfeab12f96ea2a0b0162044e1b33a\": rpc error: code = NotFound desc = could not find container \"66537becb0b66142bd959161daaa772454ddfeab12f96ea2a0b0162044e1b33a\": container with ID starting with 66537becb0b66142bd959161daaa772454ddfeab12f96ea2a0b0162044e1b33a not found: ID does not exist"
Oct 30 22:51:45.643449 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.643415    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"]
Oct 30 22:51:45.647750 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.647721    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-ae7a1f-openshift-default-68b54dc97c-kcxnh"]
Oct 30 22:51:45.960444 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:45.960409    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="167cb6f4-5f3b-4e79-bd45-3ffc48cefa33" path="/var/lib/kubelet/pods/167cb6f4-5f3b-4e79-bd45-3ffc48cefa33/volumes"
Oct 30 22:51:48.234580 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:48.234499    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:52.100672 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:52.100634    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh"]
Oct 30 22:51:52.101177 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:52.100859    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh" podUID="dbb05364-f153-4407-be11-808adf3eb888" containerName="maas-api" containerID="cri-o://2f9b6d5d2e045cbd96f52f029f9bf66479487a406fd2a6479f00e5f79361f329" gracePeriod=30
Oct 30 22:51:52.214615 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:52.214577    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:52.272626 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:52.272593    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-4d14e0-8577b8cdc7-tpcg9"]
Oct 30 22:51:52.374558 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:52.374535    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh"
Oct 30 22:51:52.452695 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:52.452657    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dbb05364-f153-4407-be11-808adf3eb888-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbb05364-f153-4407-be11-808adf3eb888-openshift-service-ca\") pod \"dbb05364-f153-4407-be11-808adf3eb888\" (UID: \"dbb05364-f153-4407-be11-808adf3eb888\") "
Oct 30 22:51:52.452896 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:52.452714    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dbb05364-f153-4407-be11-808adf3eb888-kube-api-access-9rz52\" (UniqueName: \"kubernetes.io/projected/dbb05364-f153-4407-be11-808adf3eb888-kube-api-access-9rz52\") pod \"dbb05364-f153-4407-be11-808adf3eb888\" (UID: \"dbb05364-f153-4407-be11-808adf3eb888\") "
Oct 30 22:51:52.452896 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:52.452756    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dbb05364-f153-4407-be11-808adf3eb888-metrics-certs\" (UniqueName: \"kubernetes.io/secret/dbb05364-f153-4407-be11-808adf3eb888-metrics-certs\") pod \"dbb05364-f153-4407-be11-808adf3eb888\" (UID: \"dbb05364-f153-4407-be11-808adf3eb888\") "
Oct 30 22:51:52.452896 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:52.452792    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dbb05364-f153-4407-be11-808adf3eb888-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dbb05364-f153-4407-be11-808adf3eb888-maas-api-tls\") pod \"dbb05364-f153-4407-be11-808adf3eb888\" (UID: \"dbb05364-f153-4407-be11-808adf3eb888\") "
Oct 30 22:51:52.453142 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:52.453110    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbb05364-f153-4407-be11-808adf3eb888-openshift-service-ca" pod "dbb05364-f153-4407-be11-808adf3eb888" (UID: "dbb05364-f153-4407-be11-808adf3eb888"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:51:52.455010 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:52.454978    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbb05364-f153-4407-be11-808adf3eb888-metrics-certs" pod "dbb05364-f153-4407-be11-808adf3eb888" (UID: "dbb05364-f153-4407-be11-808adf3eb888"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:51:52.455010 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:52.454992    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbb05364-f153-4407-be11-808adf3eb888-kube-api-access-9rz52" pod "dbb05364-f153-4407-be11-808adf3eb888" (UID: "dbb05364-f153-4407-be11-808adf3eb888"). InnerVolumeSpecName "kube-api-access-9rz52". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:51:52.455153 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:52.455051    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbb05364-f153-4407-be11-808adf3eb888-maas-api-tls" pod "dbb05364-f153-4407-be11-808adf3eb888" (UID: "dbb05364-f153-4407-be11-808adf3eb888"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:51:52.522399 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:52.522353    4871 generic.go:356] "Generic (PLEG): container finished" podID="dbb05364-f153-4407-be11-808adf3eb888" containerID="2f9b6d5d2e045cbd96f52f029f9bf66479487a406fd2a6479f00e5f79361f329" exitCode=0
Oct 30 22:51:52.522580 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:52.522392    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh" event={"ID":"dbb05364-f153-4407-be11-808adf3eb888","Type":"ContainerDied","Data":"2f9b6d5d2e045cbd96f52f029f9bf66479487a406fd2a6479f00e5f79361f329"}
Oct 30 22:51:52.522580 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:52.522465    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh" event={"ID":"dbb05364-f153-4407-be11-808adf3eb888","Type":"ContainerDied","Data":"80317366d900ec47f3eae7f3b16bc05668f1483c777790ba0de99588b7010a69"}
Oct 30 22:51:52.522580 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:52.522411    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh"
Oct 30 22:51:52.522580 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:52.522496    4871 scope.go:122] "RemoveContainer" containerID="2f9b6d5d2e045cbd96f52f029f9bf66479487a406fd2a6479f00e5f79361f329"
Oct 30 22:51:52.541182 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:52.541152    4871 scope.go:122] "RemoveContainer" containerID="2f9b6d5d2e045cbd96f52f029f9bf66479487a406fd2a6479f00e5f79361f329"
Oct 30 22:51:52.541641 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:51:52.541611    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f9b6d5d2e045cbd96f52f029f9bf66479487a406fd2a6479f00e5f79361f329\": container with ID starting with 2f9b6d5d2e045cbd96f52f029f9bf66479487a406fd2a6479f00e5f79361f329 not found: ID does not exist" containerID="2f9b6d5d2e045cbd96f52f029f9bf66479487a406fd2a6479f00e5f79361f329"
Oct 30 22:51:52.541732 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:52.541657    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f9b6d5d2e045cbd96f52f029f9bf66479487a406fd2a6479f00e5f79361f329"} err="failed to get container status \"2f9b6d5d2e045cbd96f52f029f9bf66479487a406fd2a6479f00e5f79361f329\": rpc error: code = NotFound desc = could not find container \"2f9b6d5d2e045cbd96f52f029f9bf66479487a406fd2a6479f00e5f79361f329\": container with ID starting with 2f9b6d5d2e045cbd96f52f029f9bf66479487a406fd2a6479f00e5f79361f329 not found: ID does not exist"
Oct 30 22:51:52.554674 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:52.554643    4871 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbb05364-f153-4407-be11-808adf3eb888-openshift-service-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:52.554803 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:52.554678    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9rz52\" (UniqueName: \"kubernetes.io/projected/dbb05364-f153-4407-be11-808adf3eb888-kube-api-access-9rz52\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:52.554803 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:52.554696    4871 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dbb05364-f153-4407-be11-808adf3eb888-metrics-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:52.554803 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:52.554712    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dbb05364-f153-4407-be11-808adf3eb888-maas-api-tls\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:52.661721 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:52.661685    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:52.662769 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:52.662742    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh"]
Oct 30 22:51:52.670108 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:52.670081    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5bb0b1-7d58fdc555-h7xfh"]
Oct 30 22:51:52.840375 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:52.838911    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:53.841266 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:53.827879    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:53.960840 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:53.960799    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dbb05364-f153-4407-be11-808adf3eb888" path="/var/lib/kubelet/pods/dbb05364-f153-4407-be11-808adf3eb888/volumes"
Oct 30 22:51:54.009593 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:54.009560    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:54.171977 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:54.171946    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:54.452321 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:54.452246    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:54.659121 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:54.659088    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:55.918175 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:55.918137    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:57.273059 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:57.273012    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w3-4d14e0-8577b8cdc7-tpcg9" podUID="d9b034ee-198d-4fcc-b726-78f720f65468" containerName="payload-processing" containerID="cri-o://e2866eb72cffce234c711621529db1ebe552e1ac1a63a546751a9b178a719d6b" gracePeriod=25
Oct 30 22:51:59.041504 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:59.041457    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:59.701369 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:51:59.701335    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:00.899634 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:00.899605    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:01.067698 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:01.067666    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:01.224181 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:01.223340    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:01.503670 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:01.503578    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:01.683367 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:01.683328    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:02.623619 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:02.623582    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:02.838267 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:02.838204    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:03.032475 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:03.032325    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:03.229907 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:03.229867    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"]
Oct 30 22:52:03.230142 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:03.230116    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt" podUID="e96f69ad-3e18-4188-9dad-67d37845519a" containerName="istio-proxy" containerID="cri-o://6636d970fa4893812ae6961d9720883bcdedc9561cd288ff6cafae9a81dc4153" gracePeriod=30
Oct 30 22:52:03.239505 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:03.239465    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:03.679687 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:03.679647    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:04.276635 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:04.276604    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:04.479420 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:04.479379    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-4d14e0-8577b8cdc7-tpcg9" podUID="d9b034ee-198d-4fcc-b726-78f720f65468" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:52:05.569497 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:52:05.569453    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:52:07.468089 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:07.468045    4871 patch_prober.go:29] interesting pod/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:52:07.470480 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:07.468114    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt" podUID="e96f69ad-3e18-4188-9dad-67d37845519a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:52:07.820908 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:07.820764    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:08.314488 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:08.314446    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:08.585980 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:08.585950    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"
Oct 30 22:52:08.644923 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:08.644893    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e96f69ad-3e18-4188-9dad-67d37845519a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e96f69ad-3e18-4188-9dad-67d37845519a-istiod-ca-cert\") pod \"e96f69ad-3e18-4188-9dad-67d37845519a\" (UID: \"e96f69ad-3e18-4188-9dad-67d37845519a\") "
Oct 30 22:52:08.645137 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:08.644934    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e96f69ad-3e18-4188-9dad-67d37845519a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e96f69ad-3e18-4188-9dad-67d37845519a-istio-envoy\") pod \"e96f69ad-3e18-4188-9dad-67d37845519a\" (UID: \"e96f69ad-3e18-4188-9dad-67d37845519a\") "
Oct 30 22:52:08.645137 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:08.644967    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e96f69ad-3e18-4188-9dad-67d37845519a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e96f69ad-3e18-4188-9dad-67d37845519a-istio-podinfo\") pod \"e96f69ad-3e18-4188-9dad-67d37845519a\" (UID: \"e96f69ad-3e18-4188-9dad-67d37845519a\") "
Oct 30 22:52:08.645137 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:08.645068    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e96f69ad-3e18-4188-9dad-67d37845519a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e96f69ad-3e18-4188-9dad-67d37845519a-istio-data\") pod \"e96f69ad-3e18-4188-9dad-67d37845519a\" (UID: \"e96f69ad-3e18-4188-9dad-67d37845519a\") "
Oct 30 22:52:08.645137 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:08.645103    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e96f69ad-3e18-4188-9dad-67d37845519a-istio-token\" (UniqueName: \"kubernetes.io/projected/e96f69ad-3e18-4188-9dad-67d37845519a-istio-token\") pod \"e96f69ad-3e18-4188-9dad-67d37845519a\" (UID: \"e96f69ad-3e18-4188-9dad-67d37845519a\") "
Oct 30 22:52:08.645422 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:08.645144    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e96f69ad-3e18-4188-9dad-67d37845519a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e96f69ad-3e18-4188-9dad-67d37845519a-workload-socket\") pod \"e96f69ad-3e18-4188-9dad-67d37845519a\" (UID: \"e96f69ad-3e18-4188-9dad-67d37845519a\") "
Oct 30 22:52:08.645422 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:08.645181    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e96f69ad-3e18-4188-9dad-67d37845519a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e96f69ad-3e18-4188-9dad-67d37845519a-workload-certs\") pod \"e96f69ad-3e18-4188-9dad-67d37845519a\" (UID: \"e96f69ad-3e18-4188-9dad-67d37845519a\") "
Oct 30 22:52:08.645422 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:08.645218    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e96f69ad-3e18-4188-9dad-67d37845519a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e96f69ad-3e18-4188-9dad-67d37845519a-credential-socket\") pod \"e96f69ad-3e18-4188-9dad-67d37845519a\" (UID: \"e96f69ad-3e18-4188-9dad-67d37845519a\") "
Oct 30 22:52:08.645422 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:08.645278    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e96f69ad-3e18-4188-9dad-67d37845519a-kube-api-access-tdxt8\" (UniqueName: \"kubernetes.io/projected/e96f69ad-3e18-4188-9dad-67d37845519a-kube-api-access-tdxt8\") pod \"e96f69ad-3e18-4188-9dad-67d37845519a\" (UID: \"e96f69ad-3e18-4188-9dad-67d37845519a\") "
Oct 30 22:52:08.648831 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:08.645899    4871 generic.go:356] "Generic (PLEG): container finished" podID="e96f69ad-3e18-4188-9dad-67d37845519a" containerID="6636d970fa4893812ae6961d9720883bcdedc9561cd288ff6cafae9a81dc4153" exitCode=0
Oct 30 22:52:08.648831 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:08.645970    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e96f69ad-3e18-4188-9dad-67d37845519a-istiod-ca-cert" pod "e96f69ad-3e18-4188-9dad-67d37845519a" (UID: "e96f69ad-3e18-4188-9dad-67d37845519a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:52:08.648831 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:08.645996    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt" event={"ID":"e96f69ad-3e18-4188-9dad-67d37845519a","Type":"ContainerDied","Data":"6636d970fa4893812ae6961d9720883bcdedc9561cd288ff6cafae9a81dc4153"}
Oct 30 22:52:08.648831 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:08.646045    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt" event={"ID":"e96f69ad-3e18-4188-9dad-67d37845519a","Type":"ContainerDied","Data":"c4ad688138744bb664ebc42ab2a4101bdaafc87fdd2a49a9afc14c3efbcbba6c"}
Oct 30 22:52:08.648831 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:08.646077    4871 scope.go:122] "RemoveContainer" containerID="6636d970fa4893812ae6961d9720883bcdedc9561cd288ff6cafae9a81dc4153"
Oct 30 22:52:08.648831 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:08.646261    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"
Oct 30 22:52:08.648831 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:08.646998    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e96f69ad-3e18-4188-9dad-67d37845519a-credential-socket" pod "e96f69ad-3e18-4188-9dad-67d37845519a" (UID: "e96f69ad-3e18-4188-9dad-67d37845519a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:52:08.648831 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:08.646947    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e96f69ad-3e18-4188-9dad-67d37845519a-workload-certs" pod "e96f69ad-3e18-4188-9dad-67d37845519a" (UID: "e96f69ad-3e18-4188-9dad-67d37845519a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:52:08.648831 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:08.647023    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e96f69ad-3e18-4188-9dad-67d37845519a-workload-socket" pod "e96f69ad-3e18-4188-9dad-67d37845519a" (UID: "e96f69ad-3e18-4188-9dad-67d37845519a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:52:08.649691 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:08.649639    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e96f69ad-3e18-4188-9dad-67d37845519a-istio-data" pod "e96f69ad-3e18-4188-9dad-67d37845519a" (UID: "e96f69ad-3e18-4188-9dad-67d37845519a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:52:08.652762 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:08.652728    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e96f69ad-3e18-4188-9dad-67d37845519a-istio-podinfo" pod "e96f69ad-3e18-4188-9dad-67d37845519a" (UID: "e96f69ad-3e18-4188-9dad-67d37845519a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:52:08.653749 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:08.653693    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e96f69ad-3e18-4188-9dad-67d37845519a-istio-token" pod "e96f69ad-3e18-4188-9dad-67d37845519a" (UID: "e96f69ad-3e18-4188-9dad-67d37845519a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:52:08.653749 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:08.653728    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e96f69ad-3e18-4188-9dad-67d37845519a-kube-api-access-tdxt8" pod "e96f69ad-3e18-4188-9dad-67d37845519a" (UID: "e96f69ad-3e18-4188-9dad-67d37845519a"). InnerVolumeSpecName "kube-api-access-tdxt8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:52:08.655357 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:08.655333    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e96f69ad-3e18-4188-9dad-67d37845519a-istio-envoy" pod "e96f69ad-3e18-4188-9dad-67d37845519a" (UID: "e96f69ad-3e18-4188-9dad-67d37845519a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:52:08.668528 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:08.668286    4871 scope.go:122] "RemoveContainer" containerID="6636d970fa4893812ae6961d9720883bcdedc9561cd288ff6cafae9a81dc4153"
Oct 30 22:52:08.668826 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:52:08.668801    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6636d970fa4893812ae6961d9720883bcdedc9561cd288ff6cafae9a81dc4153\": container with ID starting with 6636d970fa4893812ae6961d9720883bcdedc9561cd288ff6cafae9a81dc4153 not found: ID does not exist" containerID="6636d970fa4893812ae6961d9720883bcdedc9561cd288ff6cafae9a81dc4153"
Oct 30 22:52:08.668922 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:08.668839    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6636d970fa4893812ae6961d9720883bcdedc9561cd288ff6cafae9a81dc4153"} err="failed to get container status \"6636d970fa4893812ae6961d9720883bcdedc9561cd288ff6cafae9a81dc4153\": rpc error: code = NotFound desc = could not find container \"6636d970fa4893812ae6961d9720883bcdedc9561cd288ff6cafae9a81dc4153\": container with ID starting with 6636d970fa4893812ae6961d9720883bcdedc9561cd288ff6cafae9a81dc4153 not found: ID does not exist"
Oct 30 22:52:08.754522 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:08.754478    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e96f69ad-3e18-4188-9dad-67d37845519a-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:52:08.754522 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:08.754511    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e96f69ad-3e18-4188-9dad-67d37845519a-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:52:08.754522 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:08.754524    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e96f69ad-3e18-4188-9dad-67d37845519a-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:52:08.754522 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:08.754533    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e96f69ad-3e18-4188-9dad-67d37845519a-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:52:08.754929 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:08.754542    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e96f69ad-3e18-4188-9dad-67d37845519a-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:52:08.754929 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:08.754549    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e96f69ad-3e18-4188-9dad-67d37845519a-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:52:08.754929 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:08.754559    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e96f69ad-3e18-4188-9dad-67d37845519a-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:52:08.754929 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:08.754568    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e96f69ad-3e18-4188-9dad-67d37845519a-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:52:08.754929 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:08.754577    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdxt8\" (UniqueName: \"kubernetes.io/projected/e96f69ad-3e18-4188-9dad-67d37845519a-kube-api-access-tdxt8\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:52:08.843922 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:08.843842    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:09.076641 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.076596    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"]
Oct 30 22:52:09.082419 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.082388    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-5bb0b1-openshift-default-7b9f69bf65-849dt"]
Oct 30 22:52:09.299290 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.299228    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2"]
Oct 30 22:52:09.299752 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.299739    4871 cpu_manager.go:414] "containerMap: removing container" podUID="239ef6a2-b9f8-4247-9db4-fc4192f6bf1c" containerName="istio-proxy"
Oct 30 22:52:09.299797 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.299754    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="239ef6a2-b9f8-4247-9db4-fc4192f6bf1c" containerName="istio-proxy"
Oct 30 22:52:09.299797 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.299770    4871 cpu_manager.go:414] "containerMap: removing container" podUID="e96f69ad-3e18-4188-9dad-67d37845519a" containerName="istio-proxy"
Oct 30 22:52:09.299797 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.299775    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e96f69ad-3e18-4188-9dad-67d37845519a" containerName="istio-proxy"
Oct 30 22:52:09.299797 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.299783    4871 cpu_manager.go:414] "containerMap: removing container" podUID="dbb05364-f153-4407-be11-808adf3eb888" containerName="maas-api"
Oct 30 22:52:09.299797 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.299788    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dbb05364-f153-4407-be11-808adf3eb888" containerName="maas-api"
Oct 30 22:52:09.299797 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.299799    4871 cpu_manager.go:414] "containerMap: removing container" podUID="239ef6a2-b9f8-4247-9db4-fc4192f6bf1c" containerName="istio-proxy"
Oct 30 22:52:09.299995 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.299804    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="239ef6a2-b9f8-4247-9db4-fc4192f6bf1c" containerName="istio-proxy"
Oct 30 22:52:09.299995 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.299818    4871 cpu_manager.go:414] "containerMap: removing container" podUID="a9d90975-336b-49e4-b165-0b9fe73cb526" containerName="istio-proxy"
Oct 30 22:52:09.299995 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.299823    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9d90975-336b-49e4-b165-0b9fe73cb526" containerName="istio-proxy"
Oct 30 22:52:09.299995 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.299831    4871 cpu_manager.go:414] "containerMap: removing container" podUID="167cb6f4-5f3b-4e79-bd45-3ffc48cefa33" containerName="istio-proxy"
Oct 30 22:52:09.299995 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.299836    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="167cb6f4-5f3b-4e79-bd45-3ffc48cefa33" containerName="istio-proxy"
Oct 30 22:52:09.299995 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.299846    4871 cpu_manager.go:414] "containerMap: removing container" podUID="167cb6f4-5f3b-4e79-bd45-3ffc48cefa33" containerName="istio-proxy"
Oct 30 22:52:09.299995 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.299851    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="167cb6f4-5f3b-4e79-bd45-3ffc48cefa33" containerName="istio-proxy"
Oct 30 22:52:09.299995 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.299868    4871 cpu_manager.go:414] "containerMap: removing container" podUID="ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea" containerName="istio-proxy"
Oct 30 22:52:09.299995 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.299873    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ef5ef1c4-ec4f-4336-b2d4-9c727dc58bea" containerName="istio-proxy"
Oct 30 22:52:09.299995 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.299939    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="239ef6a2-b9f8-4247-9db4-fc4192f6bf1c" containerName="istio-proxy"
Oct 30 22:52:09.299995 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.299948    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="e96f69ad-3e18-4188-9dad-67d37845519a" containerName="istio-proxy"
Oct 30 22:52:09.299995 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.299953    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="dbb05364-f153-4407-be11-808adf3eb888" containerName="maas-api"
Oct 30 22:52:09.299995 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.299961    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="239ef6a2-b9f8-4247-9db4-fc4192f6bf1c" containerName="istio-proxy"
Oct 30 22:52:09.299995 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.299969    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="a9d90975-336b-49e4-b165-0b9fe73cb526" containerName="istio-proxy"
Oct 30 22:52:09.299995 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.299975    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="167cb6f4-5f3b-4e79-bd45-3ffc48cefa33" containerName="istio-proxy"
Oct 30 22:52:09.299995 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.299983    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="167cb6f4-5f3b-4e79-bd45-3ffc48cefa33" containerName="istio-proxy"
Oct 30 22:52:09.304584 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.304558    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2"
Oct 30 22:52:09.361372 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.361337    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2"]
Oct 30 22:52:09.367116 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.367084    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-9af9e7\"/\"payload-processing-external-model-e2e--287ca085-dockercfg-2hwsw\""
Oct 30 22:52:09.367701 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.367682    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-9af9e7\"/\"payload-processing-external-model-plugins-e2e-extmod-b-9af9e7\""
Oct 30 22:52:09.367772 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.367762    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-9af9e7\"/\"kube-root-ca.crt\""
Oct 30 22:52:09.367814 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.367808    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-9af9e7\"/\"routing-overlay\""
Oct 30 22:52:09.367868 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.367858    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-9af9e7\"/\"openshift-service-ca.crt\""
Oct 30 22:52:09.367909 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.367900    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-9af9e7\"/\"agc-cred-c38aa0cefc3a\""
Oct 30 22:52:09.368425 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.368403    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hgsg\" (UniqueName: \"kubernetes.io/projected/2a1963fe-96ab-4a79-a186-8d7aee904fb2-kube-api-access-6hgsg\") pod \"payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2\" (UID: \"2a1963fe-96ab-4a79-a186-8d7aee904fb2\") " pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2"
Oct 30 22:52:09.368499 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.368476    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a1963fe-96ab-4a79-a186-8d7aee904fb2-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2\" (UID: \"2a1963fe-96ab-4a79-a186-8d7aee904fb2\") " pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2"
Oct 30 22:52:09.368620 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.368604    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/2a1963fe-96ab-4a79-a186-8d7aee904fb2-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2\" (UID: \"2a1963fe-96ab-4a79-a186-8d7aee904fb2\") " pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2"
Oct 30 22:52:09.368685 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.368668    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/2a1963fe-96ab-4a79-a186-8d7aee904fb2-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2\" (UID: \"2a1963fe-96ab-4a79-a186-8d7aee904fb2\") " pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2"
Oct 30 22:52:09.402199 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.402162    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:09.470070 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.470030    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/2a1963fe-96ab-4a79-a186-8d7aee904fb2-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2\" (UID: \"2a1963fe-96ab-4a79-a186-8d7aee904fb2\") " pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2"
Oct 30 22:52:09.470070 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.470076    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hgsg\" (UniqueName: \"kubernetes.io/projected/2a1963fe-96ab-4a79-a186-8d7aee904fb2-kube-api-access-6hgsg\") pod \"payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2\" (UID: \"2a1963fe-96ab-4a79-a186-8d7aee904fb2\") " pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2"
Oct 30 22:52:09.470369 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.470128    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a1963fe-96ab-4a79-a186-8d7aee904fb2-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2\" (UID: \"2a1963fe-96ab-4a79-a186-8d7aee904fb2\") " pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2"
Oct 30 22:52:09.470369 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.470199    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/2a1963fe-96ab-4a79-a186-8d7aee904fb2-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2\" (UID: \"2a1963fe-96ab-4a79-a186-8d7aee904fb2\") " pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2"
Oct 30 22:52:09.470936 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.470906    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/2a1963fe-96ab-4a79-a186-8d7aee904fb2-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2\" (UID: \"2a1963fe-96ab-4a79-a186-8d7aee904fb2\") " pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2"
Oct 30 22:52:09.471055 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.471013    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a1963fe-96ab-4a79-a186-8d7aee904fb2-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2\" (UID: \"2a1963fe-96ab-4a79-a186-8d7aee904fb2\") " pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2"
Oct 30 22:52:09.472709 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.472687    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/2a1963fe-96ab-4a79-a186-8d7aee904fb2-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2\" (UID: \"2a1963fe-96ab-4a79-a186-8d7aee904fb2\") " pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2"
Oct 30 22:52:09.534084 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.534013    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hgsg\" (UniqueName: \"kubernetes.io/projected/2a1963fe-96ab-4a79-a186-8d7aee904fb2-kube-api-access-6hgsg\") pod \"payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2\" (UID: \"2a1963fe-96ab-4a79-a186-8d7aee904fb2\") " pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2"
Oct 30 22:52:09.620454 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.620411    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2"
Oct 30 22:52:09.960905 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:09.960872    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e96f69ad-3e18-4188-9dad-67d37845519a" path="/var/lib/kubelet/pods/e96f69ad-3e18-4188-9dad-67d37845519a/volumes"
Oct 30 22:52:10.012754 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:52:10.012709    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a1963fe_96ab_4a79_a186_8d7aee904fb2.slice/crio-029144db412cfda83e8ff24d6132c6666b3a6749213eb5bc151da75c74a52ad7 WatchSource:0}: Error finding container 029144db412cfda83e8ff24d6132c6666b3a6749213eb5bc151da75c74a52ad7: Status 404 returned error can't find the container with id 029144db412cfda83e8ff24d6132c6666b3a6749213eb5bc151da75c74a52ad7
Oct 30 22:52:10.013130 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:10.013106    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2"]
Oct 30 22:52:10.668684 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:10.668641    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2" event={"ID":"2a1963fe-96ab-4a79-a186-8d7aee904fb2","Type":"ContainerStarted","Data":"370f894a9cce3849447dd81d7b4f5df8dca865a47c84de9f50b7b55e4eda42cd"}
Oct 30 22:52:10.669162 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:10.668695    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2" event={"ID":"2a1963fe-96ab-4a79-a186-8d7aee904fb2","Type":"ContainerStarted","Data":"029144db412cfda83e8ff24d6132c6666b3a6749213eb5bc151da75c74a52ad7"}
Oct 30 22:52:10.669162 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:10.668724    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2"
Oct 30 22:52:10.795014 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:10.794955    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2" podStartSLOduration=1.79494051 podStartE2EDuration="1.79494051s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:52:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:52:10.791819193 +0000 UTC m=+4472.969307811" watchObservedRunningTime="2026-10-30 22:52:10.79494051 +0000 UTC m=+4472.972429176"
Oct 30 22:52:14.026055 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:14.026014    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:14.479225 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:14.479176    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-4d14e0-8577b8cdc7-tpcg9" podUID="d9b034ee-198d-4fcc-b726-78f720f65468" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:52:15.777322 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:15.777282    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-56f3fe-78865dc69c-22kpq"]
Oct 30 22:52:15.888938 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:15.888900    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-56f3fe-6689c845d7-c2f5w"]
Oct 30 22:52:16.153160 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:16.153126    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:16.317819 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:16.317758    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:17.575205 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:17.575166    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-4d14e0-8577b8cdc7-tpcg9"
Oct 30 22:52:17.671811 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:17.671775    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d9b034ee-198d-4fcc-b726-78f720f65468-kube-api-access-jbtqg\" (UniqueName: \"kubernetes.io/projected/d9b034ee-198d-4fcc-b726-78f720f65468-kube-api-access-jbtqg\") pod \"d9b034ee-198d-4fcc-b726-78f720f65468\" (UID: \"d9b034ee-198d-4fcc-b726-78f720f65468\") "
Oct 30 22:52:17.672016 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:17.671930    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d9b034ee-198d-4fcc-b726-78f720f65468-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9b034ee-198d-4fcc-b726-78f720f65468-plugins-config-volume\") pod \"d9b034ee-198d-4fcc-b726-78f720f65468\" (UID: \"d9b034ee-198d-4fcc-b726-78f720f65468\") "
Oct 30 22:52:17.672381 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:17.672352    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9b034ee-198d-4fcc-b726-78f720f65468-plugins-config-volume" pod "d9b034ee-198d-4fcc-b726-78f720f65468" (UID: "d9b034ee-198d-4fcc-b726-78f720f65468"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:52:17.674193 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:17.674164    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9b034ee-198d-4fcc-b726-78f720f65468-kube-api-access-jbtqg" pod "d9b034ee-198d-4fcc-b726-78f720f65468" (UID: "d9b034ee-198d-4fcc-b726-78f720f65468"). InnerVolumeSpecName "kube-api-access-jbtqg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:52:17.722267 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:17.722210    4871 generic.go:356] "Generic (PLEG): container finished" podID="d9b034ee-198d-4fcc-b726-78f720f65468" containerID="e2866eb72cffce234c711621529db1ebe552e1ac1a63a546751a9b178a719d6b" exitCode=0
Oct 30 22:52:17.722426 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:17.722274    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-4d14e0-8577b8cdc7-tpcg9"
Oct 30 22:52:17.722426 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:17.722286    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-4d14e0-8577b8cdc7-tpcg9" event={"ID":"d9b034ee-198d-4fcc-b726-78f720f65468","Type":"ContainerDied","Data":"e2866eb72cffce234c711621529db1ebe552e1ac1a63a546751a9b178a719d6b"}
Oct 30 22:52:17.722426 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:17.722389    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-4d14e0-8577b8cdc7-tpcg9" event={"ID":"d9b034ee-198d-4fcc-b726-78f720f65468","Type":"ContainerDied","Data":"ba0f395d60d417b2c3a90a2d8dd648e82f01702de6acdc25248e735fc6bd4ca1"}
Oct 30 22:52:17.722426 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:17.722424    4871 scope.go:122] "RemoveContainer" containerID="e2866eb72cffce234c711621529db1ebe552e1ac1a63a546751a9b178a719d6b"
Oct 30 22:52:17.735469 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:17.735440    4871 scope.go:122] "RemoveContainer" containerID="e2866eb72cffce234c711621529db1ebe552e1ac1a63a546751a9b178a719d6b"
Oct 30 22:52:17.735938 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:52:17.735887    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2866eb72cffce234c711621529db1ebe552e1ac1a63a546751a9b178a719d6b\": container with ID starting with e2866eb72cffce234c711621529db1ebe552e1ac1a63a546751a9b178a719d6b not found: ID does not exist" containerID="e2866eb72cffce234c711621529db1ebe552e1ac1a63a546751a9b178a719d6b"
Oct 30 22:52:17.736063 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:17.735935    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2866eb72cffce234c711621529db1ebe552e1ac1a63a546751a9b178a719d6b"} err="failed to get container status \"e2866eb72cffce234c711621529db1ebe552e1ac1a63a546751a9b178a719d6b\": rpc error: code = NotFound desc = could not find container \"e2866eb72cffce234c711621529db1ebe552e1ac1a63a546751a9b178a719d6b\": container with ID starting with e2866eb72cffce234c711621529db1ebe552e1ac1a63a546751a9b178a719d6b not found: ID does not exist"
Oct 30 22:52:17.773389 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:17.773345    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbtqg\" (UniqueName: \"kubernetes.io/projected/d9b034ee-198d-4fcc-b726-78f720f65468-kube-api-access-jbtqg\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:52:17.773389 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:17.773383    4871 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9b034ee-198d-4fcc-b726-78f720f65468-plugins-config-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:52:17.897847 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:17.897802    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-4d14e0-8577b8cdc7-tpcg9"]
Oct 30 22:52:17.902622 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:17.902591    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-4d14e0-8577b8cdc7-tpcg9"]
Oct 30 22:52:17.959903 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:17.959870    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d9b034ee-198d-4fcc-b726-78f720f65468" path="/var/lib/kubelet/pods/d9b034ee-198d-4fcc-b726-78f720f65468/volumes"
Oct 30 22:52:18.643267 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:18.643203    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:20.778416 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:20.778352    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w2-56f3fe-78865dc69c-22kpq" podUID="345d46dc-dc95-43e6-beaf-8615bfa7d148" containerName="payload-processing" containerID="cri-o://bdb92f0d739899e9fa9044f573bca7683e2fc8964501719cebfb64e5c1b0549b" gracePeriod=25
Oct 30 22:52:20.889527 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:20.889453    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-56f3fe-6689c845d7-c2f5w" podUID="5a47bd06-49c0-45df-a003-78ab3c1e2d22" containerName="payload-pre-processing" containerID="cri-o://1d175e0aa4dd8dbc0e7bb5d6eac8b0d30643d3605bd3072c44a2b8d2c00816cc" gracePeriod=25
Oct 30 22:52:21.505081 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:21.505032    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:21.682506 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:21.682469    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2"
Oct 30 22:52:21.682698 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:21.682535    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:23.157000 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:23.156967    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:24.135536 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:24.135481    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:24.402957 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:24.402845    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-56f3fe-78865dc69c-22kpq" podUID="345d46dc-dc95-43e6-beaf-8615bfa7d148" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:52:24.406497 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:24.406459    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-56f3fe-6689c845d7-c2f5w" podUID="5a47bd06-49c0-45df-a003-78ab3c1e2d22" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:52:27.917572 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:27.917527    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:29.114918 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:29.114846    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:29.260352 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:29.260316    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:31.808985 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:31.808950    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47"]
Oct 30 22:52:31.809547 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:31.809200    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47" podUID="f61a7f98-7755-4550-9f9b-fe3cce16f75f" containerName="maas-api" containerID="cri-o://3c07939c15520f06116434e4c504c8154745c454f6fcc19290fa375df072be08" gracePeriod=30
Oct 30 22:52:31.976922 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:31.976874    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:32.090871 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:32.090843    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47"
Oct 30 22:52:32.238298 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:32.238263    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f61a7f98-7755-4550-9f9b-fe3cce16f75f-kube-api-access-w9v2d\" (UniqueName: \"kubernetes.io/projected/f61a7f98-7755-4550-9f9b-fe3cce16f75f-kube-api-access-w9v2d\") pod \"f61a7f98-7755-4550-9f9b-fe3cce16f75f\" (UID: \"f61a7f98-7755-4550-9f9b-fe3cce16f75f\") "
Oct 30 22:52:32.238484 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:32.238339    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f61a7f98-7755-4550-9f9b-fe3cce16f75f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f61a7f98-7755-4550-9f9b-fe3cce16f75f-openshift-service-ca\") pod \"f61a7f98-7755-4550-9f9b-fe3cce16f75f\" (UID: \"f61a7f98-7755-4550-9f9b-fe3cce16f75f\") "
Oct 30 22:52:32.238571 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:32.238517    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f61a7f98-7755-4550-9f9b-fe3cce16f75f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f61a7f98-7755-4550-9f9b-fe3cce16f75f-metrics-certs\") pod \"f61a7f98-7755-4550-9f9b-fe3cce16f75f\" (UID: \"f61a7f98-7755-4550-9f9b-fe3cce16f75f\") "
Oct 30 22:52:32.238633 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:32.238575    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f61a7f98-7755-4550-9f9b-fe3cce16f75f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f61a7f98-7755-4550-9f9b-fe3cce16f75f-maas-api-tls\") pod \"f61a7f98-7755-4550-9f9b-fe3cce16f75f\" (UID: \"f61a7f98-7755-4550-9f9b-fe3cce16f75f\") "
Oct 30 22:52:32.239103 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:32.239068    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f61a7f98-7755-4550-9f9b-fe3cce16f75f-openshift-service-ca" pod "f61a7f98-7755-4550-9f9b-fe3cce16f75f" (UID: "f61a7f98-7755-4550-9f9b-fe3cce16f75f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:52:32.240950 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:32.240915    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f61a7f98-7755-4550-9f9b-fe3cce16f75f-metrics-certs" pod "f61a7f98-7755-4550-9f9b-fe3cce16f75f" (UID: "f61a7f98-7755-4550-9f9b-fe3cce16f75f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:52:32.241101 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:32.241076    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f61a7f98-7755-4550-9f9b-fe3cce16f75f-maas-api-tls" pod "f61a7f98-7755-4550-9f9b-fe3cce16f75f" (UID: "f61a7f98-7755-4550-9f9b-fe3cce16f75f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:52:32.241307 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:32.241276    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f61a7f98-7755-4550-9f9b-fe3cce16f75f-kube-api-access-w9v2d" pod "f61a7f98-7755-4550-9f9b-fe3cce16f75f" (UID: "f61a7f98-7755-4550-9f9b-fe3cce16f75f"). InnerVolumeSpecName "kube-api-access-w9v2d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:52:32.340671 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:32.340572    4871 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f61a7f98-7755-4550-9f9b-fe3cce16f75f-metrics-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:52:32.340671 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:32.340610    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f61a7f98-7755-4550-9f9b-fe3cce16f75f-maas-api-tls\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:52:32.340671 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:32.340625    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w9v2d\" (UniqueName: \"kubernetes.io/projected/f61a7f98-7755-4550-9f9b-fe3cce16f75f-kube-api-access-w9v2d\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:52:32.340671 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:32.340639    4871 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f61a7f98-7755-4550-9f9b-fe3cce16f75f-openshift-service-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:52:32.823324 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:32.823290    4871 generic.go:356] "Generic (PLEG): container finished" podID="f61a7f98-7755-4550-9f9b-fe3cce16f75f" containerID="3c07939c15520f06116434e4c504c8154745c454f6fcc19290fa375df072be08" exitCode=0
Oct 30 22:52:32.823911 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:32.823345    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47"
Oct 30 22:52:32.823911 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:32.823386    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47" event={"ID":"f61a7f98-7755-4550-9f9b-fe3cce16f75f","Type":"ContainerDied","Data":"3c07939c15520f06116434e4c504c8154745c454f6fcc19290fa375df072be08"}
Oct 30 22:52:32.823911 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:32.823455    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47" event={"ID":"f61a7f98-7755-4550-9f9b-fe3cce16f75f","Type":"ContainerDied","Data":"cf4bfe958ddafae63dbc9e8a9ce9b9c0660bf6b2a8b3f993f94c3f400c0c0990"}
Oct 30 22:52:32.823911 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:32.823492    4871 scope.go:122] "RemoveContainer" containerID="3c07939c15520f06116434e4c504c8154745c454f6fcc19290fa375df072be08"
Oct 30 22:52:32.834597 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:32.834578    4871 scope.go:122] "RemoveContainer" containerID="3c07939c15520f06116434e4c504c8154745c454f6fcc19290fa375df072be08"
Oct 30 22:52:32.834873 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:52:32.834854    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c07939c15520f06116434e4c504c8154745c454f6fcc19290fa375df072be08\": container with ID starting with 3c07939c15520f06116434e4c504c8154745c454f6fcc19290fa375df072be08 not found: ID does not exist" containerID="3c07939c15520f06116434e4c504c8154745c454f6fcc19290fa375df072be08"
Oct 30 22:52:32.834956 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:32.834881    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c07939c15520f06116434e4c504c8154745c454f6fcc19290fa375df072be08"} err="failed to get container status \"3c07939c15520f06116434e4c504c8154745c454f6fcc19290fa375df072be08\": rpc error: code = NotFound desc = could not find container \"3c07939c15520f06116434e4c504c8154745c454f6fcc19290fa375df072be08\": container with ID starting with 3c07939c15520f06116434e4c504c8154745c454f6fcc19290fa375df072be08 not found: ID does not exist"
Oct 30 22:52:32.997990 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:32.997949    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47"]
Oct 30 22:52:33.003691 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:33.003650    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-4d14e0-f7c55dc66-dwn47"]
Oct 30 22:52:33.388817 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:33.388785    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:33.594719 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:33.594685    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:33.745753 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:33.745672    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:33.959702 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:33.959669    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f61a7f98-7755-4550-9f9b-fe3cce16f75f" path="/var/lib/kubelet/pods/f61a7f98-7755-4550-9f9b-fe3cce16f75f/volumes"
Oct 30 22:52:34.325800 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:34.325770    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:34.402718 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:34.402674    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-56f3fe-6689c845d7-c2f5w" podUID="5a47bd06-49c0-45df-a003-78ab3c1e2d22" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:52:34.402916 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:34.402749    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-56f3fe-78865dc69c-22kpq" podUID="345d46dc-dc95-43e6-beaf-8615bfa7d148" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:52:35.568207 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:52:35.568171    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:52:38.614548 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:38.614513    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:38.889363 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:38.889282    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx_8d401654-8a4c-4a79-b2d3-612765312421/extract/0.log"
Oct 30 22:52:38.903498 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:38.903469    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx_8d401654-8a4c-4a79-b2d3-612765312421/extract/0.log"
Oct 30 22:52:38.905565 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:38.905538    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/2.log"
Oct 30 22:52:38.913267 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:38.913221    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzntd_2360fddc-52a5-42a5-82ca-a05aff84d217/ovn-acl-logging/0.log"
Oct 30 22:52:38.916586 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:38.916564    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/2.log"
Oct 30 22:52:38.923940 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:38.923921    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzntd_2360fddc-52a5-42a5-82ca-a05aff84d217/ovn-acl-logging/0.log"
Oct 30 22:52:40.746399 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:40.746359    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:40.909701 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:40.909664    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:41.107338 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:41.107310    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-56f3fe-78865dc69c-22kpq"
Oct 30 22:52:41.119703 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:41.119640    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:41.125832 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:41.125803    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"]
Oct 30 22:52:41.126016 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:41.125997    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx" podUID="1969a36c-2ac9-4972-8a9e-e84e39682a28" containerName="istio-proxy" containerID="cri-o://d22346676e75efcd76c9470ac6e977700c4c39d07cdcffc3438a067bf23a5dae" gracePeriod=30
Oct 30 22:52:41.132811 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:41.132784    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/345d46dc-dc95-43e6-beaf-8615bfa7d148-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/345d46dc-dc95-43e6-beaf-8615bfa7d148-plugins-config-volume\") pod \"345d46dc-dc95-43e6-beaf-8615bfa7d148\" (UID: \"345d46dc-dc95-43e6-beaf-8615bfa7d148\") "
Oct 30 22:52:41.132986 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:41.132938    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/345d46dc-dc95-43e6-beaf-8615bfa7d148-kube-api-access-2zfm8\" (UniqueName: \"kubernetes.io/projected/345d46dc-dc95-43e6-beaf-8615bfa7d148-kube-api-access-2zfm8\") pod \"345d46dc-dc95-43e6-beaf-8615bfa7d148\" (UID: \"345d46dc-dc95-43e6-beaf-8615bfa7d148\") "
Oct 30 22:52:41.133214 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:41.133186    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/345d46dc-dc95-43e6-beaf-8615bfa7d148-plugins-config-volume" pod "345d46dc-dc95-43e6-beaf-8615bfa7d148" (UID: "345d46dc-dc95-43e6-beaf-8615bfa7d148"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:52:41.135758 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:41.135728    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/345d46dc-dc95-43e6-beaf-8615bfa7d148-kube-api-access-2zfm8" pod "345d46dc-dc95-43e6-beaf-8615bfa7d148" (UID: "345d46dc-dc95-43e6-beaf-8615bfa7d148"). InnerVolumeSpecName "kube-api-access-2zfm8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:52:41.166107 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:41.166081    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-56f3fe-6689c845d7-c2f5w"
Oct 30 22:52:41.234020 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:41.233908    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5a47bd06-49c0-45df-a003-78ab3c1e2d22-kube-api-access-zfs6b\" (UniqueName: \"kubernetes.io/projected/5a47bd06-49c0-45df-a003-78ab3c1e2d22-kube-api-access-zfs6b\") pod \"5a47bd06-49c0-45df-a003-78ab3c1e2d22\" (UID: \"5a47bd06-49c0-45df-a003-78ab3c1e2d22\") "
Oct 30 22:52:41.234020 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:41.234003    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5a47bd06-49c0-45df-a003-78ab3c1e2d22-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a47bd06-49c0-45df-a003-78ab3c1e2d22-plugins-config-volume\") pod \"5a47bd06-49c0-45df-a003-78ab3c1e2d22\" (UID: \"5a47bd06-49c0-45df-a003-78ab3c1e2d22\") "
Oct 30 22:52:41.234362 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:41.234344    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2zfm8\" (UniqueName: \"kubernetes.io/projected/345d46dc-dc95-43e6-beaf-8615bfa7d148-kube-api-access-2zfm8\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:52:41.234436 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:41.234366    4871 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/345d46dc-dc95-43e6-beaf-8615bfa7d148-plugins-config-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:52:41.234759 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:41.234735    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a47bd06-49c0-45df-a003-78ab3c1e2d22-plugins-config-volume" pod "5a47bd06-49c0-45df-a003-78ab3c1e2d22" (UID: "5a47bd06-49c0-45df-a003-78ab3c1e2d22"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:52:41.238270 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:41.237435    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a47bd06-49c0-45df-a003-78ab3c1e2d22-kube-api-access-zfs6b" pod "5a47bd06-49c0-45df-a003-78ab3c1e2d22" (UID: "5a47bd06-49c0-45df-a003-78ab3c1e2d22"). InnerVolumeSpecName "kube-api-access-zfs6b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:52:41.335059 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:41.334975    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zfs6b\" (UniqueName: \"kubernetes.io/projected/5a47bd06-49c0-45df-a003-78ab3c1e2d22-kube-api-access-zfs6b\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:52:41.335059 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:41.335004    4871 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a47bd06-49c0-45df-a003-78ab3c1e2d22-plugins-config-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:52:41.638635 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:41.638594    4871 scope.go:122] "RemoveContainer" containerID="bdb92f0d739899e9fa9044f573bca7683e2fc8964501719cebfb64e5c1b0549b"
Oct 30 22:52:41.649141 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:41.649112    4871 scope.go:122] "RemoveContainer" containerID="1d175e0aa4dd8dbc0e7bb5d6eac8b0d30643d3605bd3072c44a2b8d2c00816cc"
Oct 30 22:52:41.883317 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:41.883275    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-56f3fe-6689c845d7-c2f5w" event={"ID":"5a47bd06-49c0-45df-a003-78ab3c1e2d22","Type":"ContainerDied","Data":"1d175e0aa4dd8dbc0e7bb5d6eac8b0d30643d3605bd3072c44a2b8d2c00816cc"}
Oct 30 22:52:41.883779 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:41.883355    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-56f3fe-6689c845d7-c2f5w" event={"ID":"5a47bd06-49c0-45df-a003-78ab3c1e2d22","Type":"ContainerDied","Data":"f6786a4eda806742b63c25236fbbdc7bfe803373b6e30833e0a793973372ba76"}
Oct 30 22:52:41.883779 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:41.883384    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-56f3fe-78865dc69c-22kpq" event={"ID":"345d46dc-dc95-43e6-beaf-8615bfa7d148","Type":"ContainerDied","Data":"bdb92f0d739899e9fa9044f573bca7683e2fc8964501719cebfb64e5c1b0549b"}
Oct 30 22:52:41.883779 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:41.883409    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-56f3fe-78865dc69c-22kpq" event={"ID":"345d46dc-dc95-43e6-beaf-8615bfa7d148","Type":"ContainerDied","Data":"7fb08e3a8b03c15fa485aa04ba4975f1999fd9d3a00b72e34282cc78411b51de"}
Oct 30 22:52:41.883779 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:41.883437    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-56f3fe-6689c845d7-c2f5w"
Oct 30 22:52:41.883779 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:41.883441    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-56f3fe-78865dc69c-22kpq"
Oct 30 22:52:42.189549 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:42.189443    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-56f3fe-78865dc69c-22kpq"]
Oct 30 22:52:42.194145 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:42.194112    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-56f3fe-78865dc69c-22kpq"]
Oct 30 22:52:42.308082 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:42.308046    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-56f3fe-6689c845d7-c2f5w"]
Oct 30 22:52:42.310842 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:42.310811    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-56f3fe-6689c845d7-c2f5w"]
Oct 30 22:52:42.615744 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:42.615705    4871 patch_prober.go:29] interesting pod/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:52:42.615954 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:42.615772    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx" podUID="1969a36c-2ac9-4972-8a9e-e84e39682a28" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:52:43.215947 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:43.215911    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:43.959400 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:43.959361    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="345d46dc-dc95-43e6-beaf-8615bfa7d148" path="/var/lib/kubelet/pods/345d46dc-dc95-43e6-beaf-8615bfa7d148/volumes"
Oct 30 22:52:43.959829 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:43.959812    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5a47bd06-49c0-45df-a003-78ab3c1e2d22" path="/var/lib/kubelet/pods/5a47bd06-49c0-45df-a003-78ab3c1e2d22/volumes"
Oct 30 22:52:46.183168 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.183139    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:46.344668 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.344628    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:46.472802 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.472773    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"
Oct 30 22:52:46.490428 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.490395    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1969a36c-2ac9-4972-8a9e-e84e39682a28-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1969a36c-2ac9-4972-8a9e-e84e39682a28-istio-podinfo\") pod \"1969a36c-2ac9-4972-8a9e-e84e39682a28\" (UID: \"1969a36c-2ac9-4972-8a9e-e84e39682a28\") "
Oct 30 22:52:46.490616 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.490454    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1969a36c-2ac9-4972-8a9e-e84e39682a28-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1969a36c-2ac9-4972-8a9e-e84e39682a28-credential-socket\") pod \"1969a36c-2ac9-4972-8a9e-e84e39682a28\" (UID: \"1969a36c-2ac9-4972-8a9e-e84e39682a28\") "
Oct 30 22:52:46.490616 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.490482    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1969a36c-2ac9-4972-8a9e-e84e39682a28-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1969a36c-2ac9-4972-8a9e-e84e39682a28-workload-certs\") pod \"1969a36c-2ac9-4972-8a9e-e84e39682a28\" (UID: \"1969a36c-2ac9-4972-8a9e-e84e39682a28\") "
Oct 30 22:52:46.490616 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.490524    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1969a36c-2ac9-4972-8a9e-e84e39682a28-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1969a36c-2ac9-4972-8a9e-e84e39682a28-workload-socket\") pod \"1969a36c-2ac9-4972-8a9e-e84e39682a28\" (UID: \"1969a36c-2ac9-4972-8a9e-e84e39682a28\") "
Oct 30 22:52:46.490616 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.490563    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1969a36c-2ac9-4972-8a9e-e84e39682a28-kube-api-access-4bblw\" (UniqueName: \"kubernetes.io/projected/1969a36c-2ac9-4972-8a9e-e84e39682a28-kube-api-access-4bblw\") pod \"1969a36c-2ac9-4972-8a9e-e84e39682a28\" (UID: \"1969a36c-2ac9-4972-8a9e-e84e39682a28\") "
Oct 30 22:52:46.490616 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.490591    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1969a36c-2ac9-4972-8a9e-e84e39682a28-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1969a36c-2ac9-4972-8a9e-e84e39682a28-istiod-ca-cert\") pod \"1969a36c-2ac9-4972-8a9e-e84e39682a28\" (UID: \"1969a36c-2ac9-4972-8a9e-e84e39682a28\") "
Oct 30 22:52:46.490887 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.490616    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1969a36c-2ac9-4972-8a9e-e84e39682a28-istio-token\" (UniqueName: \"kubernetes.io/projected/1969a36c-2ac9-4972-8a9e-e84e39682a28-istio-token\") pod \"1969a36c-2ac9-4972-8a9e-e84e39682a28\" (UID: \"1969a36c-2ac9-4972-8a9e-e84e39682a28\") "
Oct 30 22:52:46.490887 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.490645    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1969a36c-2ac9-4972-8a9e-e84e39682a28-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1969a36c-2ac9-4972-8a9e-e84e39682a28-istio-data\") pod \"1969a36c-2ac9-4972-8a9e-e84e39682a28\" (UID: \"1969a36c-2ac9-4972-8a9e-e84e39682a28\") "
Oct 30 22:52:46.490887 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.490728    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1969a36c-2ac9-4972-8a9e-e84e39682a28-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1969a36c-2ac9-4972-8a9e-e84e39682a28-istio-envoy\") pod \"1969a36c-2ac9-4972-8a9e-e84e39682a28\" (UID: \"1969a36c-2ac9-4972-8a9e-e84e39682a28\") "
Oct 30 22:52:46.490887 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.490811    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1969a36c-2ac9-4972-8a9e-e84e39682a28-workload-certs" pod "1969a36c-2ac9-4972-8a9e-e84e39682a28" (UID: "1969a36c-2ac9-4972-8a9e-e84e39682a28"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:52:46.490887 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.490824    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1969a36c-2ac9-4972-8a9e-e84e39682a28-credential-socket" pod "1969a36c-2ac9-4972-8a9e-e84e39682a28" (UID: "1969a36c-2ac9-4972-8a9e-e84e39682a28"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:52:46.491139 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.491116    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1969a36c-2ac9-4972-8a9e-e84e39682a28-istio-data" pod "1969a36c-2ac9-4972-8a9e-e84e39682a28" (UID: "1969a36c-2ac9-4972-8a9e-e84e39682a28"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:52:46.491292 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.491142    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1969a36c-2ac9-4972-8a9e-e84e39682a28-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:52:46.491292 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.491162    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1969a36c-2ac9-4972-8a9e-e84e39682a28-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:52:46.491292 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.491136    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1969a36c-2ac9-4972-8a9e-e84e39682a28-istiod-ca-cert" pod "1969a36c-2ac9-4972-8a9e-e84e39682a28" (UID: "1969a36c-2ac9-4972-8a9e-e84e39682a28"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:52:46.491292 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.491173    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1969a36c-2ac9-4972-8a9e-e84e39682a28-workload-socket" pod "1969a36c-2ac9-4972-8a9e-e84e39682a28" (UID: "1969a36c-2ac9-4972-8a9e-e84e39682a28"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:52:46.493178 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.493150    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1969a36c-2ac9-4972-8a9e-e84e39682a28-istio-podinfo" pod "1969a36c-2ac9-4972-8a9e-e84e39682a28" (UID: "1969a36c-2ac9-4972-8a9e-e84e39682a28"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:52:46.493342 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.493178    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1969a36c-2ac9-4972-8a9e-e84e39682a28-istio-token" pod "1969a36c-2ac9-4972-8a9e-e84e39682a28" (UID: "1969a36c-2ac9-4972-8a9e-e84e39682a28"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:52:46.493598 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.493571    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1969a36c-2ac9-4972-8a9e-e84e39682a28-istio-envoy" pod "1969a36c-2ac9-4972-8a9e-e84e39682a28" (UID: "1969a36c-2ac9-4972-8a9e-e84e39682a28"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:52:46.493598 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.493589    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1969a36c-2ac9-4972-8a9e-e84e39682a28-kube-api-access-4bblw" pod "1969a36c-2ac9-4972-8a9e-e84e39682a28" (UID: "1969a36c-2ac9-4972-8a9e-e84e39682a28"). InnerVolumeSpecName "kube-api-access-4bblw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:52:46.592540 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.592503    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1969a36c-2ac9-4972-8a9e-e84e39682a28-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:52:46.592540 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.592534    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1969a36c-2ac9-4972-8a9e-e84e39682a28-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:52:46.592769 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.592548    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1969a36c-2ac9-4972-8a9e-e84e39682a28-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:52:46.592769 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.592561    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bblw\" (UniqueName: \"kubernetes.io/projected/1969a36c-2ac9-4972-8a9e-e84e39682a28-kube-api-access-4bblw\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:52:46.592769 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.592570    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1969a36c-2ac9-4972-8a9e-e84e39682a28-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:52:46.592769 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.592579    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1969a36c-2ac9-4972-8a9e-e84e39682a28-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:52:46.592769 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.592588    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1969a36c-2ac9-4972-8a9e-e84e39682a28-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:52:46.608954 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.608918    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:46.892147 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.892110    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:46.927295 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.927220    4871 generic.go:356] "Generic (PLEG): container finished" podID="1969a36c-2ac9-4972-8a9e-e84e39682a28" containerID="d22346676e75efcd76c9470ac6e977700c4c39d07cdcffc3438a067bf23a5dae" exitCode=0
Oct 30 22:52:46.927512 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.927377    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx" event={"ID":"1969a36c-2ac9-4972-8a9e-e84e39682a28","Type":"ContainerDied","Data":"d22346676e75efcd76c9470ac6e977700c4c39d07cdcffc3438a067bf23a5dae"}
Oct 30 22:52:46.927512 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.927434    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx" event={"ID":"1969a36c-2ac9-4972-8a9e-e84e39682a28","Type":"ContainerDied","Data":"3a18e13917f1ef78563593db17981e4deaaf32825130a0051e15c0a76c8a8d08"}
Oct 30 22:52:46.927512 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.927464    4871 scope.go:122] "RemoveContainer" containerID="d22346676e75efcd76c9470ac6e977700c4c39d07cdcffc3438a067bf23a5dae"
Oct 30 22:52:46.927701 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.927628    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"
Oct 30 22:52:46.940146 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.940126    4871 scope.go:122] "RemoveContainer" containerID="cb13b4186f0a942b24f7f268d5138b480ab6df06de9ac90c33879029c3618849"
Oct 30 22:52:46.951472 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.951447    4871 scope.go:122] "RemoveContainer" containerID="d22346676e75efcd76c9470ac6e977700c4c39d07cdcffc3438a067bf23a5dae"
Oct 30 22:52:46.951830 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:52:46.951806    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d22346676e75efcd76c9470ac6e977700c4c39d07cdcffc3438a067bf23a5dae\": container with ID starting with d22346676e75efcd76c9470ac6e977700c4c39d07cdcffc3438a067bf23a5dae not found: ID does not exist" containerID="d22346676e75efcd76c9470ac6e977700c4c39d07cdcffc3438a067bf23a5dae"
Oct 30 22:52:46.951937 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.951841    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d22346676e75efcd76c9470ac6e977700c4c39d07cdcffc3438a067bf23a5dae"} err="failed to get container status \"d22346676e75efcd76c9470ac6e977700c4c39d07cdcffc3438a067bf23a5dae\": rpc error: code = NotFound desc = could not find container \"d22346676e75efcd76c9470ac6e977700c4c39d07cdcffc3438a067bf23a5dae\": container with ID starting with d22346676e75efcd76c9470ac6e977700c4c39d07cdcffc3438a067bf23a5dae not found: ID does not exist"
Oct 30 22:52:46.951937 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.951861    4871 scope.go:122] "RemoveContainer" containerID="cb13b4186f0a942b24f7f268d5138b480ab6df06de9ac90c33879029c3618849"
Oct 30 22:52:46.952137 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:52:46.952114    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb13b4186f0a942b24f7f268d5138b480ab6df06de9ac90c33879029c3618849\": container with ID starting with cb13b4186f0a942b24f7f268d5138b480ab6df06de9ac90c33879029c3618849 not found: ID does not exist" containerID="cb13b4186f0a942b24f7f268d5138b480ab6df06de9ac90c33879029c3618849"
Oct 30 22:52:46.952201 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:46.952144    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb13b4186f0a942b24f7f268d5138b480ab6df06de9ac90c33879029c3618849"} err="failed to get container status \"cb13b4186f0a942b24f7f268d5138b480ab6df06de9ac90c33879029c3618849\": rpc error: code = NotFound desc = could not find container \"cb13b4186f0a942b24f7f268d5138b480ab6df06de9ac90c33879029c3618849\": container with ID starting with cb13b4186f0a942b24f7f268d5138b480ab6df06de9ac90c33879029c3618849 not found: ID does not exist"
Oct 30 22:52:47.106158 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:47.106119    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"]
Oct 30 22:52:47.110617 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:47.110576    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-4d14e0-openshift-default-649bbfdc7f-zrdwx"]
Oct 30 22:52:47.841252 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:47.841202    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:47.962188 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:47.962143    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1969a36c-2ac9-4972-8a9e-e84e39682a28" path="/var/lib/kubelet/pods/1969a36c-2ac9-4972-8a9e-e84e39682a28/volumes"
Oct 30 22:52:52.555650 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:52.555611    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:53.161918 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:53.161871    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:54.809273 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:54.809222    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:56.313210 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:56.313176    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:57.135257 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:57.135199    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:59.156859 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:59.156807    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:59.309616 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:52:59.309582    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:01.612779 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:01.612742    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:05.568478 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:53:05.568437    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:53:06.255823 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:06.255784    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:10.209425 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:10.209390    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:11.833178 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:11.833120    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:11.989338 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:11.989304    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:12.141339 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:12.141305    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:12.680089 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:12.680054    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:13.424631 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:13.424591    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:13.583804 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:13.583773    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:16.834494 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:16.834456    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:17.479376 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:17.479338    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5"]
Oct 30 22:53:17.479803 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:17.479790    4871 cpu_manager.go:414] "containerMap: removing container" podUID="5a47bd06-49c0-45df-a003-78ab3c1e2d22" containerName="payload-pre-processing"
Oct 30 22:53:17.479853 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:17.479804    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5a47bd06-49c0-45df-a003-78ab3c1e2d22" containerName="payload-pre-processing"
Oct 30 22:53:17.479853 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:17.479817    4871 cpu_manager.go:414] "containerMap: removing container" podUID="1969a36c-2ac9-4972-8a9e-e84e39682a28" containerName="istio-proxy"
Oct 30 22:53:17.479853 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:17.479824    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1969a36c-2ac9-4972-8a9e-e84e39682a28" containerName="istio-proxy"
Oct 30 22:53:17.479853 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:17.479832    4871 cpu_manager.go:414] "containerMap: removing container" podUID="d9b034ee-198d-4fcc-b726-78f720f65468" containerName="payload-processing"
Oct 30 22:53:17.479853 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:17.479837    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d9b034ee-198d-4fcc-b726-78f720f65468" containerName="payload-processing"
Oct 30 22:53:17.480020 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:17.479857    4871 cpu_manager.go:414] "containerMap: removing container" podUID="1969a36c-2ac9-4972-8a9e-e84e39682a28" containerName="istio-proxy"
Oct 30 22:53:17.480020 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:17.479863    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1969a36c-2ac9-4972-8a9e-e84e39682a28" containerName="istio-proxy"
Oct 30 22:53:17.480020 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:17.479878    4871 cpu_manager.go:414] "containerMap: removing container" podUID="345d46dc-dc95-43e6-beaf-8615bfa7d148" containerName="payload-processing"
Oct 30 22:53:17.480020 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:17.479883    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="345d46dc-dc95-43e6-beaf-8615bfa7d148" containerName="payload-processing"
Oct 30 22:53:17.480020 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:17.479900    4871 cpu_manager.go:414] "containerMap: removing container" podUID="f61a7f98-7755-4550-9f9b-fe3cce16f75f" containerName="maas-api"
Oct 30 22:53:17.480020 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:17.479904    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f61a7f98-7755-4550-9f9b-fe3cce16f75f" containerName="maas-api"
Oct 30 22:53:17.480020 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:17.479964    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="f61a7f98-7755-4550-9f9b-fe3cce16f75f" containerName="maas-api"
Oct 30 22:53:17.480020 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:17.479973    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="5a47bd06-49c0-45df-a003-78ab3c1e2d22" containerName="payload-pre-processing"
Oct 30 22:53:17.480020 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:17.479980    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="1969a36c-2ac9-4972-8a9e-e84e39682a28" containerName="istio-proxy"
Oct 30 22:53:17.480020 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:17.479986    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="d9b034ee-198d-4fcc-b726-78f720f65468" containerName="payload-processing"
Oct 30 22:53:17.480020 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:17.479995    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="1969a36c-2ac9-4972-8a9e-e84e39682a28" containerName="istio-proxy"
Oct 30 22:53:17.480020 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:17.480004    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="345d46dc-dc95-43e6-beaf-8615bfa7d148" containerName="payload-processing"
Oct 30 22:53:17.488654 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:17.488613    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5"
Oct 30 22:53:17.539375 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:17.539338    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5"]
Oct 30 22:53:17.587839 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:17.587804    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-9af9e7\"/\"agc-cred-9103685eeaf8\""
Oct 30 22:53:17.629584 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:17.629543    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb52525e-ca08-4be0-afc0-954385b1b723-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5\" (UID: \"cb52525e-ca08-4be0-afc0-954385b1b723\") " pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5"
Oct 30 22:53:17.629799 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:17.629605    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6znk6\" (UniqueName: \"kubernetes.io/projected/cb52525e-ca08-4be0-afc0-954385b1b723-kube-api-access-6znk6\") pod \"payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5\" (UID: \"cb52525e-ca08-4be0-afc0-954385b1b723\") " pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5"
Oct 30 22:53:17.629799 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:17.629735    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/cb52525e-ca08-4be0-afc0-954385b1b723-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5\" (UID: \"cb52525e-ca08-4be0-afc0-954385b1b723\") " pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5"
Oct 30 22:53:17.629887 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:17.629809    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/cb52525e-ca08-4be0-afc0-954385b1b723-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5\" (UID: \"cb52525e-ca08-4be0-afc0-954385b1b723\") " pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5"
Oct 30 22:53:17.731046 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:17.730950    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb52525e-ca08-4be0-afc0-954385b1b723-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5\" (UID: \"cb52525e-ca08-4be0-afc0-954385b1b723\") " pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5"
Oct 30 22:53:17.731046 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:17.730994    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6znk6\" (UniqueName: \"kubernetes.io/projected/cb52525e-ca08-4be0-afc0-954385b1b723-kube-api-access-6znk6\") pod \"payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5\" (UID: \"cb52525e-ca08-4be0-afc0-954385b1b723\") " pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5"
Oct 30 22:53:17.731046 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:17.731019    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/cb52525e-ca08-4be0-afc0-954385b1b723-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5\" (UID: \"cb52525e-ca08-4be0-afc0-954385b1b723\") " pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5"
Oct 30 22:53:17.731420 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:17.731064    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/cb52525e-ca08-4be0-afc0-954385b1b723-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5\" (UID: \"cb52525e-ca08-4be0-afc0-954385b1b723\") " pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5"
Oct 30 22:53:17.731846 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:17.731818    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/cb52525e-ca08-4be0-afc0-954385b1b723-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5\" (UID: \"cb52525e-ca08-4be0-afc0-954385b1b723\") " pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5"
Oct 30 22:53:17.732006 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:17.731966    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb52525e-ca08-4be0-afc0-954385b1b723-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5\" (UID: \"cb52525e-ca08-4be0-afc0-954385b1b723\") " pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5"
Oct 30 22:53:17.733876 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:17.733846    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/cb52525e-ca08-4be0-afc0-954385b1b723-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5\" (UID: \"cb52525e-ca08-4be0-afc0-954385b1b723\") " pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5"
Oct 30 22:53:17.796717 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:17.796685    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6znk6\" (UniqueName: \"kubernetes.io/projected/cb52525e-ca08-4be0-afc0-954385b1b723-kube-api-access-6znk6\") pod \"payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5\" (UID: \"cb52525e-ca08-4be0-afc0-954385b1b723\") " pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5"
Oct 30 22:53:17.804425 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:17.804396    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5"
Oct 30 22:53:18.185730 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:53:18.185680    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb52525e_ca08_4be0_afc0_954385b1b723.slice/crio-bc987b47f1414202a24d3faf7104ba7fc7a492aa4e872a1bea97324a9a6e9303 WatchSource:0}: Error finding container bc987b47f1414202a24d3faf7104ba7fc7a492aa4e872a1bea97324a9a6e9303: Status 404 returned error can't find the container with id bc987b47f1414202a24d3faf7104ba7fc7a492aa4e872a1bea97324a9a6e9303
Oct 30 22:53:18.189556 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:18.189531    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5"]
Oct 30 22:53:19.139134 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:19.139095    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5" event={"ID":"cb52525e-ca08-4be0-afc0-954385b1b723","Type":"ContainerStarted","Data":"acad62a2c631e6f6b65971462f0f53ed9de6b4c98226e2cb53d25fb2e1331e4a"}
Oct 30 22:53:19.139378 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:19.139148    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5" event={"ID":"cb52525e-ca08-4be0-afc0-954385b1b723","Type":"ContainerStarted","Data":"bc987b47f1414202a24d3faf7104ba7fc7a492aa4e872a1bea97324a9a6e9303"}
Oct 30 22:53:19.139378 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:19.139260    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5"
Oct 30 22:53:19.265328 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:19.265277    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5" podStartSLOduration=2.265262563 podStartE2EDuration="2.265262563s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:53:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:53:19.26174793 +0000 UTC m=+4541.439236548" watchObservedRunningTime="2026-10-30 22:53:19.265262563 +0000 UTC m=+4541.442751179"
Oct 30 22:53:19.809294 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:19.809257    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:20.045983 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:20.045940    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:20.233698 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:20.233659    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:20.428509 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:20.428477    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:20.946548 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:20.946509    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-a-0c4786/payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r"]
Oct 30 22:53:20.950628 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:20.950594    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-0c4786/payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r"
Oct 30 22:53:20.970993 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:20.970964    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/406a92fa-f298-4c04-9cde-4bdc43c9c385-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r\" (UID: \"406a92fa-f298-4c04-9cde-4bdc43c9c385\") " pod="ai-tenant-e2e-extmod-a-0c4786/payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r"
Oct 30 22:53:20.971147 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:20.971020    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/406a92fa-f298-4c04-9cde-4bdc43c9c385-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r\" (UID: \"406a92fa-f298-4c04-9cde-4bdc43c9c385\") " pod="ai-tenant-e2e-extmod-a-0c4786/payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r"
Oct 30 22:53:20.971147 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:20.971091    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/406a92fa-f298-4c04-9cde-4bdc43c9c385-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r\" (UID: \"406a92fa-f298-4c04-9cde-4bdc43c9c385\") " pod="ai-tenant-e2e-extmod-a-0c4786/payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r"
Oct 30 22:53:20.971230 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:20.971152    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rg7q\" (UniqueName: \"kubernetes.io/projected/406a92fa-f298-4c04-9cde-4bdc43c9c385-kube-api-access-6rg7q\") pod \"payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r\" (UID: \"406a92fa-f298-4c04-9cde-4bdc43c9c385\") " pod="ai-tenant-e2e-extmod-a-0c4786/payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r"
Oct 30 22:53:21.006481 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:21.006450    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-a-0c4786/payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r"]
Oct 30 22:53:21.054728 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:21.054690    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-0c4786\"/\"payload-processing-external-model-plugins-e2e-extmod-a-0c4786\""
Oct 30 22:53:21.054932 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:21.054741    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-0c4786\"/\"openshift-service-ca.crt\""
Oct 30 22:53:21.055008 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:21.054951    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-0c4786\"/\"kube-root-ca.crt\""
Oct 30 22:53:21.055136 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:21.055118    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-0c4786\"/\"routing-overlay\""
Oct 30 22:53:21.084168 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:21.084123    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/406a92fa-f298-4c04-9cde-4bdc43c9c385-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r\" (UID: \"406a92fa-f298-4c04-9cde-4bdc43c9c385\") " pod="ai-tenant-e2e-extmod-a-0c4786/payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r"
Oct 30 22:53:21.084559 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:21.084532    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/406a92fa-f298-4c04-9cde-4bdc43c9c385-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r\" (UID: \"406a92fa-f298-4c04-9cde-4bdc43c9c385\") " pod="ai-tenant-e2e-extmod-a-0c4786/payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r"
Oct 30 22:53:21.084719 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:21.084703    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/406a92fa-f298-4c04-9cde-4bdc43c9c385-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r\" (UID: \"406a92fa-f298-4c04-9cde-4bdc43c9c385\") " pod="ai-tenant-e2e-extmod-a-0c4786/payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r"
Oct 30 22:53:21.084901 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:21.084883    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rg7q\" (UniqueName: \"kubernetes.io/projected/406a92fa-f298-4c04-9cde-4bdc43c9c385-kube-api-access-6rg7q\") pod \"payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r\" (UID: \"406a92fa-f298-4c04-9cde-4bdc43c9c385\") " pod="ai-tenant-e2e-extmod-a-0c4786/payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r"
Oct 30 22:53:21.085210 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:21.085181    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/406a92fa-f298-4c04-9cde-4bdc43c9c385-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r\" (UID: \"406a92fa-f298-4c04-9cde-4bdc43c9c385\") " pod="ai-tenant-e2e-extmod-a-0c4786/payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r"
Oct 30 22:53:21.086193 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:21.086038    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/406a92fa-f298-4c04-9cde-4bdc43c9c385-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r\" (UID: \"406a92fa-f298-4c04-9cde-4bdc43c9c385\") " pod="ai-tenant-e2e-extmod-a-0c4786/payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r"
Oct 30 22:53:21.086832 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:21.086800    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-a-0c4786\"/\"agc-cred-04899ad79755\""
Oct 30 22:53:21.103924 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:21.103894    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/406a92fa-f298-4c04-9cde-4bdc43c9c385-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r\" (UID: \"406a92fa-f298-4c04-9cde-4bdc43c9c385\") " pod="ai-tenant-e2e-extmod-a-0c4786/payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r"
Oct 30 22:53:21.149366 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:21.149310    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rg7q\" (UniqueName: \"kubernetes.io/projected/406a92fa-f298-4c04-9cde-4bdc43c9c385-kube-api-access-6rg7q\") pod \"payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r\" (UID: \"406a92fa-f298-4c04-9cde-4bdc43c9c385\") " pod="ai-tenant-e2e-extmod-a-0c4786/payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r"
Oct 30 22:53:21.265199 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:21.265104    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-0c4786/payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r"
Oct 30 22:53:21.644968 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:53:21.644875    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod406a92fa_f298_4c04_9cde_4bdc43c9c385.slice/crio-13f42dd952d8ab8814589e4b8924f4436d09c43dcaa59f014d2ed6078d1054fa WatchSource:0}: Error finding container 13f42dd952d8ab8814589e4b8924f4436d09c43dcaa59f014d2ed6078d1054fa: Status 404 returned error can't find the container with id 13f42dd952d8ab8814589e4b8924f4436d09c43dcaa59f014d2ed6078d1054fa
Oct 30 22:53:21.650427 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:21.649793    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-a-0c4786/payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r"]
Oct 30 22:53:21.736993 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:21.736963    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:22.022676 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:22.022581    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:22.161620 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:22.161575    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-0c4786/payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r" event={"ID":"406a92fa-f298-4c04-9cde-4bdc43c9c385","Type":"ContainerStarted","Data":"d42802420e12eaec8fe756049bbe0da45feb7ea0194ce31424e4ec7805c56bd6"}
Oct 30 22:53:22.161802 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:22.161653    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-a-0c4786/payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r"
Oct 30 22:53:22.161802 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:22.161678    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-0c4786/payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r" event={"ID":"406a92fa-f298-4c04-9cde-4bdc43c9c385","Type":"ContainerStarted","Data":"13f42dd952d8ab8814589e4b8924f4436d09c43dcaa59f014d2ed6078d1054fa"}
Oct 30 22:53:22.196515 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:53:22.196474    4871 projected.go:266] Couldn't get secret ai-tenant-e2e-extmod-a-0c4786/agc-cred-04899ad79755: secret "agc-cred-04899ad79755" not found
Oct 30 22:53:22.196515 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:53:22.196503    4871 projected.go:196] Error preparing data for projected volume provider-credentials for pod ai-tenant-e2e-extmod-a-0c4786/payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r: secret "agc-cred-04899ad79755" not found
Oct 30 22:53:22.196800 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:53:22.196563    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/406a92fa-f298-4c04-9cde-4bdc43c9c385-provider-credentials podName:406a92fa-f298-4c04-9cde-4bdc43c9c385 nodeName:}" failed. No retries permitted until 2026-10-30 22:53:22.696543461 +0000 UTC m=+4544.874032075 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/406a92fa-f298-4c04-9cde-4bdc43c9c385-provider-credentials") pod "payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r" (UID: "406a92fa-f298-4c04-9cde-4bdc43c9c385") : secret "agc-cred-04899ad79755" not found
Oct 30 22:53:22.201505 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:22.201479    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:22.284848 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:22.284739    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-a-0c4786/payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r" podStartSLOduration=2.28472349 podStartE2EDuration="2.28472349s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:53:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:53:22.283494752 +0000 UTC m=+4544.460983370" watchObservedRunningTime="2026-10-30 22:53:22.28472349 +0000 UTC m=+4544.462212106"
Oct 30 22:53:22.702336 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:53:22.702298    4871 projected.go:266] Couldn't get secret ai-tenant-e2e-extmod-a-0c4786/agc-cred-04899ad79755: secret "agc-cred-04899ad79755" not found
Oct 30 22:53:22.702336 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:53:22.702330    4871 projected.go:196] Error preparing data for projected volume provider-credentials for pod ai-tenant-e2e-extmod-a-0c4786/payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r: secret "agc-cred-04899ad79755" not found
Oct 30 22:53:22.702794 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:53:22.702390    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/406a92fa-f298-4c04-9cde-4bdc43c9c385-provider-credentials podName:406a92fa-f298-4c04-9cde-4bdc43c9c385 nodeName:}" failed. No retries permitted until 2026-10-30 22:53:23.7023708 +0000 UTC m=+4545.879859397 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/406a92fa-f298-4c04-9cde-4bdc43c9c385-provider-credentials") pod "payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r" (UID: "406a92fa-f298-4c04-9cde-4bdc43c9c385") : secret "agc-cred-04899ad79755" not found
Oct 30 22:53:22.935068 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:22.935027    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5"]
Oct 30 22:53:23.711706 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:53:23.711655    4871 projected.go:266] Couldn't get secret ai-tenant-e2e-extmod-a-0c4786/agc-cred-04899ad79755: secret "agc-cred-04899ad79755" not found
Oct 30 22:53:23.711706 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:53:23.711691    4871 projected.go:196] Error preparing data for projected volume provider-credentials for pod ai-tenant-e2e-extmod-a-0c4786/payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r: secret "agc-cred-04899ad79755" not found
Oct 30 22:53:23.712371 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:53:23.711770    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/406a92fa-f298-4c04-9cde-4bdc43c9c385-provider-credentials podName:406a92fa-f298-4c04-9cde-4bdc43c9c385 nodeName:}" failed. No retries permitted until 2026-10-30 22:53:25.711751606 +0000 UTC m=+4547.889240219 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/406a92fa-f298-4c04-9cde-4bdc43c9c385-provider-credentials") pod "payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r" (UID: "406a92fa-f298-4c04-9cde-4bdc43c9c385") : secret "agc-cred-04899ad79755" not found
Oct 30 22:53:24.432625 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:24.432585    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-a-0c4786/payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r"]
Oct 30 22:53:24.586278 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:24.586218    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:25.736631 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:53:25.736599    4871 projected.go:266] Couldn't get secret ai-tenant-e2e-extmod-a-0c4786/agc-cred-04899ad79755: secret "agc-cred-04899ad79755" not found
Oct 30 22:53:25.736631 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:53:25.736627    4871 projected.go:196] Error preparing data for projected volume provider-credentials for pod ai-tenant-e2e-extmod-a-0c4786/payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r: secret "agc-cred-04899ad79755" not found
Oct 30 22:53:25.737150 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:53:25.736685    4871 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/406a92fa-f298-4c04-9cde-4bdc43c9c385-provider-credentials podName:406a92fa-f298-4c04-9cde-4bdc43c9c385 nodeName:}" failed. No retries permitted until 2026-10-30 22:53:29.736666949 +0000 UTC m=+4551.914155546 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/406a92fa-f298-4c04-9cde-4bdc43c9c385-provider-credentials") pod "payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r" (UID: "406a92fa-f298-4c04-9cde-4bdc43c9c385") : secret "agc-cred-04899ad79755" not found
Oct 30 22:53:26.304642 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:26.304611    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:27.936289 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:27.936182    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5" podUID="cb52525e-ca08-4be0-afc0-954385b1b723" containerName="payload-processing" containerID="cri-o://acad62a2c631e6f6b65971462f0f53ed9de6b4c98226e2cb53d25fb2e1331e4a" gracePeriod=25
Oct 30 22:53:28.199147 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:28.199078    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5"
Oct 30 22:53:28.199933 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:28.199908    4871 generic.go:356] "Generic (PLEG): container finished" podID="cb52525e-ca08-4be0-afc0-954385b1b723" containerID="acad62a2c631e6f6b65971462f0f53ed9de6b4c98226e2cb53d25fb2e1331e4a" exitCode=0
Oct 30 22:53:28.200040 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:28.199975    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5" event={"ID":"cb52525e-ca08-4be0-afc0-954385b1b723","Type":"ContainerDied","Data":"acad62a2c631e6f6b65971462f0f53ed9de6b4c98226e2cb53d25fb2e1331e4a"}
Oct 30 22:53:28.200040 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:28.200024    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5" event={"ID":"cb52525e-ca08-4be0-afc0-954385b1b723","Type":"ContainerDied","Data":"bc987b47f1414202a24d3faf7104ba7fc7a492aa4e872a1bea97324a9a6e9303"}
Oct 30 22:53:28.200127 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:28.200047    4871 scope.go:122] "RemoveContainer" containerID="acad62a2c631e6f6b65971462f0f53ed9de6b4c98226e2cb53d25fb2e1331e4a"
Oct 30 22:53:28.211496 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:28.211478    4871 scope.go:122] "RemoveContainer" containerID="acad62a2c631e6f6b65971462f0f53ed9de6b4c98226e2cb53d25fb2e1331e4a"
Oct 30 22:53:28.211825 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:53:28.211795    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"acad62a2c631e6f6b65971462f0f53ed9de6b4c98226e2cb53d25fb2e1331e4a\": container with ID starting with acad62a2c631e6f6b65971462f0f53ed9de6b4c98226e2cb53d25fb2e1331e4a not found: ID does not exist" containerID="acad62a2c631e6f6b65971462f0f53ed9de6b4c98226e2cb53d25fb2e1331e4a"
Oct 30 22:53:28.211922 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:28.211832    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acad62a2c631e6f6b65971462f0f53ed9de6b4c98226e2cb53d25fb2e1331e4a"} err="failed to get container status \"acad62a2c631e6f6b65971462f0f53ed9de6b4c98226e2cb53d25fb2e1331e4a\": rpc error: code = NotFound desc = could not find container \"acad62a2c631e6f6b65971462f0f53ed9de6b4c98226e2cb53d25fb2e1331e4a\": container with ID starting with acad62a2c631e6f6b65971462f0f53ed9de6b4c98226e2cb53d25fb2e1331e4a not found: ID does not exist"
Oct 30 22:53:28.262674 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:28.262640    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cb52525e-ca08-4be0-afc0-954385b1b723-kube-api-access-6znk6\" (UniqueName: \"kubernetes.io/projected/cb52525e-ca08-4be0-afc0-954385b1b723-kube-api-access-6znk6\") pod \"cb52525e-ca08-4be0-afc0-954385b1b723\" (UID: \"cb52525e-ca08-4be0-afc0-954385b1b723\") "
Oct 30 22:53:28.262875 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:28.262707    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cb52525e-ca08-4be0-afc0-954385b1b723-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/cb52525e-ca08-4be0-afc0-954385b1b723-routing-overlay-volume\") pod \"cb52525e-ca08-4be0-afc0-954385b1b723\" (UID: \"cb52525e-ca08-4be0-afc0-954385b1b723\") "
Oct 30 22:53:28.262956 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:28.262871    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cb52525e-ca08-4be0-afc0-954385b1b723-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb52525e-ca08-4be0-afc0-954385b1b723-plugins-config-volume\") pod \"cb52525e-ca08-4be0-afc0-954385b1b723\" (UID: \"cb52525e-ca08-4be0-afc0-954385b1b723\") "
Oct 30 22:53:28.262956 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:28.262922    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cb52525e-ca08-4be0-afc0-954385b1b723-provider-credentials\" (UniqueName: \"kubernetes.io/projected/cb52525e-ca08-4be0-afc0-954385b1b723-provider-credentials\") pod \"cb52525e-ca08-4be0-afc0-954385b1b723\" (UID: \"cb52525e-ca08-4be0-afc0-954385b1b723\") "
Oct 30 22:53:28.263193 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:28.263162    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb52525e-ca08-4be0-afc0-954385b1b723-routing-overlay-volume" pod "cb52525e-ca08-4be0-afc0-954385b1b723" (UID: "cb52525e-ca08-4be0-afc0-954385b1b723"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:53:28.263343 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:28.263316    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb52525e-ca08-4be0-afc0-954385b1b723-plugins-config-volume" pod "cb52525e-ca08-4be0-afc0-954385b1b723" (UID: "cb52525e-ca08-4be0-afc0-954385b1b723"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:53:28.263486 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:28.263463    4871 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/cb52525e-ca08-4be0-afc0-954385b1b723-routing-overlay-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:53:28.263486 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:28.263487    4871 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb52525e-ca08-4be0-afc0-954385b1b723-plugins-config-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:53:28.265082 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:28.265050    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb52525e-ca08-4be0-afc0-954385b1b723-provider-credentials" pod "cb52525e-ca08-4be0-afc0-954385b1b723" (UID: "cb52525e-ca08-4be0-afc0-954385b1b723"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:53:28.265197 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:28.265102    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb52525e-ca08-4be0-afc0-954385b1b723-kube-api-access-6znk6" pod "cb52525e-ca08-4be0-afc0-954385b1b723" (UID: "cb52525e-ca08-4be0-afc0-954385b1b723"). InnerVolumeSpecName "kube-api-access-6znk6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:53:28.365172 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:28.365134    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6znk6\" (UniqueName: \"kubernetes.io/projected/cb52525e-ca08-4be0-afc0-954385b1b723-kube-api-access-6znk6\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:53:28.365172 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:28.365169    4871 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/cb52525e-ca08-4be0-afc0-954385b1b723-provider-credentials\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:53:29.207919 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:29.207884    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5"
Oct 30 22:53:29.381800 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:29.381764    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5"]
Oct 30 22:53:29.433410 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:29.433356    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-a-0c4786/payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r" podUID="406a92fa-f298-4c04-9cde-4bdc43c9c385" containerName="payload-processing" containerID="cri-o://d42802420e12eaec8fe756049bbe0da45feb7ea0194ce31424e4ec7805c56bd6" gracePeriod=25
Oct 30 22:53:29.434472 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:29.434451    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-55842d5j5"]
Oct 30 22:53:29.697526 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:29.697496    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-0c4786/payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r"
Oct 30 22:53:29.782730 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:29.782644    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/406a92fa-f298-4c04-9cde-4bdc43c9c385-kube-api-access-6rg7q\" (UniqueName: \"kubernetes.io/projected/406a92fa-f298-4c04-9cde-4bdc43c9c385-kube-api-access-6rg7q\") pod \"406a92fa-f298-4c04-9cde-4bdc43c9c385\" (UID: \"406a92fa-f298-4c04-9cde-4bdc43c9c385\") "
Oct 30 22:53:29.782914 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:29.782748    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/406a92fa-f298-4c04-9cde-4bdc43c9c385-provider-credentials\" (UniqueName: \"kubernetes.io/projected/406a92fa-f298-4c04-9cde-4bdc43c9c385-provider-credentials\") pod \"406a92fa-f298-4c04-9cde-4bdc43c9c385\" (UID: \"406a92fa-f298-4c04-9cde-4bdc43c9c385\") "
Oct 30 22:53:29.782914 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:29.782768    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/406a92fa-f298-4c04-9cde-4bdc43c9c385-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/406a92fa-f298-4c04-9cde-4bdc43c9c385-routing-overlay-volume\") pod \"406a92fa-f298-4c04-9cde-4bdc43c9c385\" (UID: \"406a92fa-f298-4c04-9cde-4bdc43c9c385\") "
Oct 30 22:53:29.782914 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:29.782805    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/406a92fa-f298-4c04-9cde-4bdc43c9c385-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/406a92fa-f298-4c04-9cde-4bdc43c9c385-plugins-config-volume\") pod \"406a92fa-f298-4c04-9cde-4bdc43c9c385\" (UID: \"406a92fa-f298-4c04-9cde-4bdc43c9c385\") "
Oct 30 22:53:29.783195 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:29.783162    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/406a92fa-f298-4c04-9cde-4bdc43c9c385-routing-overlay-volume" pod "406a92fa-f298-4c04-9cde-4bdc43c9c385" (UID: "406a92fa-f298-4c04-9cde-4bdc43c9c385"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:53:29.783311 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:29.783291    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/406a92fa-f298-4c04-9cde-4bdc43c9c385-plugins-config-volume" pod "406a92fa-f298-4c04-9cde-4bdc43c9c385" (UID: "406a92fa-f298-4c04-9cde-4bdc43c9c385"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:53:29.784996 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:29.784975    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/406a92fa-f298-4c04-9cde-4bdc43c9c385-kube-api-access-6rg7q" pod "406a92fa-f298-4c04-9cde-4bdc43c9c385" (UID: "406a92fa-f298-4c04-9cde-4bdc43c9c385"). InnerVolumeSpecName "kube-api-access-6rg7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:53:29.785075 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:29.785007    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/406a92fa-f298-4c04-9cde-4bdc43c9c385-provider-credentials" pod "406a92fa-f298-4c04-9cde-4bdc43c9c385" (UID: "406a92fa-f298-4c04-9cde-4bdc43c9c385"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:53:29.884594 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:29.884557    4871 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/406a92fa-f298-4c04-9cde-4bdc43c9c385-plugins-config-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:53:29.884594 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:29.884586    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rg7q\" (UniqueName: \"kubernetes.io/projected/406a92fa-f298-4c04-9cde-4bdc43c9c385-kube-api-access-6rg7q\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:53:29.884594 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:29.884596    4871 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/406a92fa-f298-4c04-9cde-4bdc43c9c385-provider-credentials\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:53:29.884594 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:29.884605    4871 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/406a92fa-f298-4c04-9cde-4bdc43c9c385-routing-overlay-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:53:29.959946 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:29.959909    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cb52525e-ca08-4be0-afc0-954385b1b723" path="/var/lib/kubelet/pods/cb52525e-ca08-4be0-afc0-954385b1b723/volumes"
Oct 30 22:53:30.215803 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:30.215765    4871 generic.go:356] "Generic (PLEG): container finished" podID="406a92fa-f298-4c04-9cde-4bdc43c9c385" containerID="d42802420e12eaec8fe756049bbe0da45feb7ea0194ce31424e4ec7805c56bd6" exitCode=0
Oct 30 22:53:30.216290 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:30.215811    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-0c4786/payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r"
Oct 30 22:53:30.216290 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:30.215848    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-0c4786/payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r" event={"ID":"406a92fa-f298-4c04-9cde-4bdc43c9c385","Type":"ContainerDied","Data":"d42802420e12eaec8fe756049bbe0da45feb7ea0194ce31424e4ec7805c56bd6"}
Oct 30 22:53:30.216290 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:30.215910    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-0c4786/payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r" event={"ID":"406a92fa-f298-4c04-9cde-4bdc43c9c385","Type":"ContainerDied","Data":"13f42dd952d8ab8814589e4b8924f4436d09c43dcaa59f014d2ed6078d1054fa"}
Oct 30 22:53:30.216290 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:30.215943    4871 scope.go:122] "RemoveContainer" containerID="d42802420e12eaec8fe756049bbe0da45feb7ea0194ce31424e4ec7805c56bd6"
Oct 30 22:53:30.226219 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:30.226192    4871 scope.go:122] "RemoveContainer" containerID="d42802420e12eaec8fe756049bbe0da45feb7ea0194ce31424e4ec7805c56bd6"
Oct 30 22:53:30.226530 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:53:30.226504    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d42802420e12eaec8fe756049bbe0da45feb7ea0194ce31424e4ec7805c56bd6\": container with ID starting with d42802420e12eaec8fe756049bbe0da45feb7ea0194ce31424e4ec7805c56bd6 not found: ID does not exist" containerID="d42802420e12eaec8fe756049bbe0da45feb7ea0194ce31424e4ec7805c56bd6"
Oct 30 22:53:30.226635 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:30.226537    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d42802420e12eaec8fe756049bbe0da45feb7ea0194ce31424e4ec7805c56bd6"} err="failed to get container status \"d42802420e12eaec8fe756049bbe0da45feb7ea0194ce31424e4ec7805c56bd6\": rpc error: code = NotFound desc = could not find container \"d42802420e12eaec8fe756049bbe0da45feb7ea0194ce31424e4ec7805c56bd6\": container with ID starting with d42802420e12eaec8fe756049bbe0da45feb7ea0194ce31424e4ec7805c56bd6 not found: ID does not exist"
Oct 30 22:53:30.389185 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:30.389148    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-a-0c4786/payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r"]
Oct 30 22:53:30.392579 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:30.392551    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-a-0c4786/payload-processing-external-model-e2e-extmod-a-0c4786-5bc6d2b4r"]
Oct 30 22:53:30.911874 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:30.911835    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:31.959706 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:31.959667    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="406a92fa-f298-4c04-9cde-4bdc43c9c385" path="/var/lib/kubelet/pods/406a92fa-f298-4c04-9cde-4bdc43c9c385/volumes"
Oct 30 22:53:35.506495 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:35.506443    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:35.578960 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:53:35.578923    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:53:37.359501 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:37.359470    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:37.377441 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:37.377410    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2"]
Oct 30 22:53:37.515068 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:37.515010    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:37.666064 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:37.666030    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:38.136785 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:38.136747    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:40.211418 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:40.211382    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:42.378003 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:42.377952    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2" podUID="2a1963fe-96ab-4a79-a186-8d7aee904fb2" containerName="payload-processing" containerID="cri-o://370f894a9cce3849447dd81d7b4f5df8dca865a47c84de9f50b7b55e4eda42cd" gracePeriod=25
Oct 30 22:53:42.643231 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:42.643166    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2"
Oct 30 22:53:42.714558 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:42.714515    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2a1963fe-96ab-4a79-a186-8d7aee904fb2-provider-credentials\" (UniqueName: \"kubernetes.io/projected/2a1963fe-96ab-4a79-a186-8d7aee904fb2-provider-credentials\") pod \"2a1963fe-96ab-4a79-a186-8d7aee904fb2\" (UID: \"2a1963fe-96ab-4a79-a186-8d7aee904fb2\") "
Oct 30 22:53:42.714777 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:42.714740    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2a1963fe-96ab-4a79-a186-8d7aee904fb2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a1963fe-96ab-4a79-a186-8d7aee904fb2-plugins-config-volume\") pod \"2a1963fe-96ab-4a79-a186-8d7aee904fb2\" (UID: \"2a1963fe-96ab-4a79-a186-8d7aee904fb2\") "
Oct 30 22:53:42.714858 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:42.714793    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2a1963fe-96ab-4a79-a186-8d7aee904fb2-kube-api-access-6hgsg\" (UniqueName: \"kubernetes.io/projected/2a1963fe-96ab-4a79-a186-8d7aee904fb2-kube-api-access-6hgsg\") pod \"2a1963fe-96ab-4a79-a186-8d7aee904fb2\" (UID: \"2a1963fe-96ab-4a79-a186-8d7aee904fb2\") "
Oct 30 22:53:42.714858 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:42.714822    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2a1963fe-96ab-4a79-a186-8d7aee904fb2-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/2a1963fe-96ab-4a79-a186-8d7aee904fb2-routing-overlay-volume\") pod \"2a1963fe-96ab-4a79-a186-8d7aee904fb2\" (UID: \"2a1963fe-96ab-4a79-a186-8d7aee904fb2\") "
Oct 30 22:53:42.715315 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:42.715273    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a1963fe-96ab-4a79-a186-8d7aee904fb2-plugins-config-volume" pod "2a1963fe-96ab-4a79-a186-8d7aee904fb2" (UID: "2a1963fe-96ab-4a79-a186-8d7aee904fb2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:53:42.715461 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:42.715305    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a1963fe-96ab-4a79-a186-8d7aee904fb2-routing-overlay-volume" pod "2a1963fe-96ab-4a79-a186-8d7aee904fb2" (UID: "2a1963fe-96ab-4a79-a186-8d7aee904fb2"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:53:42.716889 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:42.716861    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a1963fe-96ab-4a79-a186-8d7aee904fb2-provider-credentials" pod "2a1963fe-96ab-4a79-a186-8d7aee904fb2" (UID: "2a1963fe-96ab-4a79-a186-8d7aee904fb2"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:53:42.717078 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:42.717054    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a1963fe-96ab-4a79-a186-8d7aee904fb2-kube-api-access-6hgsg" pod "2a1963fe-96ab-4a79-a186-8d7aee904fb2" (UID: "2a1963fe-96ab-4a79-a186-8d7aee904fb2"). InnerVolumeSpecName "kube-api-access-6hgsg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:53:42.816425 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:42.816382    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hgsg\" (UniqueName: \"kubernetes.io/projected/2a1963fe-96ab-4a79-a186-8d7aee904fb2-kube-api-access-6hgsg\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:53:42.816425 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:42.816428    4871 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/2a1963fe-96ab-4a79-a186-8d7aee904fb2-routing-overlay-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:53:42.816646 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:42.816445    4871 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/2a1963fe-96ab-4a79-a186-8d7aee904fb2-provider-credentials\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:53:42.816646 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:42.816459    4871 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a1963fe-96ab-4a79-a186-8d7aee904fb2-plugins-config-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:53:43.299530 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:43.299492    4871 generic.go:356] "Generic (PLEG): container finished" podID="2a1963fe-96ab-4a79-a186-8d7aee904fb2" containerID="370f894a9cce3849447dd81d7b4f5df8dca865a47c84de9f50b7b55e4eda42cd" exitCode=0
Oct 30 22:53:43.299734 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:43.299536    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2" event={"ID":"2a1963fe-96ab-4a79-a186-8d7aee904fb2","Type":"ContainerDied","Data":"370f894a9cce3849447dd81d7b4f5df8dca865a47c84de9f50b7b55e4eda42cd"}
Oct 30 22:53:43.299734 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:43.299576    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2" event={"ID":"2a1963fe-96ab-4a79-a186-8d7aee904fb2","Type":"ContainerDied","Data":"029144db412cfda83e8ff24d6132c6666b3a6749213eb5bc151da75c74a52ad7"}
Oct 30 22:53:43.299734 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:43.299597    4871 scope.go:122] "RemoveContainer" containerID="370f894a9cce3849447dd81d7b4f5df8dca865a47c84de9f50b7b55e4eda42cd"
Oct 30 22:53:43.299734 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:43.299539    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2"
Oct 30 22:53:43.310105 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:43.310085    4871 scope.go:122] "RemoveContainer" containerID="370f894a9cce3849447dd81d7b4f5df8dca865a47c84de9f50b7b55e4eda42cd"
Oct 30 22:53:43.310508 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:53:43.310485    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"370f894a9cce3849447dd81d7b4f5df8dca865a47c84de9f50b7b55e4eda42cd\": container with ID starting with 370f894a9cce3849447dd81d7b4f5df8dca865a47c84de9f50b7b55e4eda42cd not found: ID does not exist" containerID="370f894a9cce3849447dd81d7b4f5df8dca865a47c84de9f50b7b55e4eda42cd"
Oct 30 22:53:43.310627 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:43.310524    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"370f894a9cce3849447dd81d7b4f5df8dca865a47c84de9f50b7b55e4eda42cd"} err="failed to get container status \"370f894a9cce3849447dd81d7b4f5df8dca865a47c84de9f50b7b55e4eda42cd\": rpc error: code = NotFound desc = could not find container \"370f894a9cce3849447dd81d7b4f5df8dca865a47c84de9f50b7b55e4eda42cd\": container with ID starting with 370f894a9cce3849447dd81d7b4f5df8dca865a47c84de9f50b7b55e4eda42cd not found: ID does not exist"
Oct 30 22:53:43.486283 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:43.484034    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2"]
Oct 30 22:53:43.534607 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:43.534571    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-9af9e7/payload-processing-external-model-e2e-extmod-b-9af9e7-7746rfsd2"]
Oct 30 22:53:43.959475 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:43.959439    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2a1963fe-96ab-4a79-a186-8d7aee904fb2" path="/var/lib/kubelet/pods/2a1963fe-96ab-4a79-a186-8d7aee904fb2/volumes"
Oct 30 22:53:44.364409 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:44.364372    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:44.522672 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:44.522638    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:44.682410 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:44.682327    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:44.845891 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:44.845855    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:49.416187 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:49.416118    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:51.046848 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:51.046818    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:51.314443 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:51.314337    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:54.311764 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:54.311715    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:55.146811 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:55.146773    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:55.348759 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:55.348720    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:56.604148 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:56.604113    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:59.201842 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:53:59.201807    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:03.536995 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:03.536961    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:03.695619 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:03.695585    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:03.849389 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:03.849306    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:05.569329 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:54:05.569283    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:54:08.341712 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:08.341654    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:12.913223 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:12.913181    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:16.949021 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:16.948976    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:18.409597 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:18.409553    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:18.612425 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:18.612386    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:18.774583 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:18.774486    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:19.352621 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:19.352583    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:23.471509 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:23.471470    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:25.677720 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:25.677690    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:25.894693 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:25.894655    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:26.094957 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:26.094822    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:28.239736 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:28.239691    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:33.121651 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:33.121608    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:35.568000 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:54:35.567957    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:54:37.734607 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:37.734571    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:38.859613 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:38.859574    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:39.025430 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:39.025396    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:40.729534 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:40.729479    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:41.706216 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:41.706181    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf"]
Oct 30 22:54:41.706421 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:41.706386    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf" podUID="bb586958-b1df-4396-8ca4-f2bbf71addbc" containerName="maas-api" containerID="cri-o://4d08f2354fbca3486078415b96e42d8f4305db0c998c0a934500a6bfe8e3b3b6" gracePeriod=30
Oct 30 22:54:41.822282 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:41.822221    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:41.978006 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:41.977933    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf"
Oct 30 22:54:42.029752 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:42.029702    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bb586958-b1df-4396-8ca4-f2bbf71addbc-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb586958-b1df-4396-8ca4-f2bbf71addbc-maas-api-tls\") pod \"bb586958-b1df-4396-8ca4-f2bbf71addbc\" (UID: \"bb586958-b1df-4396-8ca4-f2bbf71addbc\") "
Oct 30 22:54:42.029970 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:42.029788    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bb586958-b1df-4396-8ca4-f2bbf71addbc-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb586958-b1df-4396-8ca4-f2bbf71addbc-openshift-service-ca\") pod \"bb586958-b1df-4396-8ca4-f2bbf71addbc\" (UID: \"bb586958-b1df-4396-8ca4-f2bbf71addbc\") "
Oct 30 22:54:42.029970 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:42.029814    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bb586958-b1df-4396-8ca4-f2bbf71addbc-metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb586958-b1df-4396-8ca4-f2bbf71addbc-metrics-certs\") pod \"bb586958-b1df-4396-8ca4-f2bbf71addbc\" (UID: \"bb586958-b1df-4396-8ca4-f2bbf71addbc\") "
Oct 30 22:54:42.029970 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:42.029935    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bb586958-b1df-4396-8ca4-f2bbf71addbc-kube-api-access-vvhwz\" (UniqueName: \"kubernetes.io/projected/bb586958-b1df-4396-8ca4-f2bbf71addbc-kube-api-access-vvhwz\") pod \"bb586958-b1df-4396-8ca4-f2bbf71addbc\" (UID: \"bb586958-b1df-4396-8ca4-f2bbf71addbc\") "
Oct 30 22:54:42.030173 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:42.030149    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb586958-b1df-4396-8ca4-f2bbf71addbc-openshift-service-ca" pod "bb586958-b1df-4396-8ca4-f2bbf71addbc" (UID: "bb586958-b1df-4396-8ca4-f2bbf71addbc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:54:42.030357 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:42.030335    4871 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb586958-b1df-4396-8ca4-f2bbf71addbc-openshift-service-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:54:42.031968 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:42.031942    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb586958-b1df-4396-8ca4-f2bbf71addbc-maas-api-tls" pod "bb586958-b1df-4396-8ca4-f2bbf71addbc" (UID: "bb586958-b1df-4396-8ca4-f2bbf71addbc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:54:42.031968 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:42.031960    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb586958-b1df-4396-8ca4-f2bbf71addbc-metrics-certs" pod "bb586958-b1df-4396-8ca4-f2bbf71addbc" (UID: "bb586958-b1df-4396-8ca4-f2bbf71addbc"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:54:42.032098 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:42.031983    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb586958-b1df-4396-8ca4-f2bbf71addbc-kube-api-access-vvhwz" pod "bb586958-b1df-4396-8ca4-f2bbf71addbc" (UID: "bb586958-b1df-4396-8ca4-f2bbf71addbc"). InnerVolumeSpecName "kube-api-access-vvhwz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:54:42.131191 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:42.131152    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb586958-b1df-4396-8ca4-f2bbf71addbc-maas-api-tls\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:54:42.131191 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:42.131187    4871 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb586958-b1df-4396-8ca4-f2bbf71addbc-metrics-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:54:42.131427 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:42.131199    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvhwz\" (UniqueName: \"kubernetes.io/projected/bb586958-b1df-4396-8ca4-f2bbf71addbc-kube-api-access-vvhwz\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:54:42.665900 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:42.665864    4871 generic.go:356] "Generic (PLEG): container finished" podID="bb586958-b1df-4396-8ca4-f2bbf71addbc" containerID="4d08f2354fbca3486078415b96e42d8f4305db0c998c0a934500a6bfe8e3b3b6" exitCode=0
Oct 30 22:54:42.666092 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:42.665912    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf"
Oct 30 22:54:42.666092 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:42.665947    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf" event={"ID":"bb586958-b1df-4396-8ca4-f2bbf71addbc","Type":"ContainerDied","Data":"4d08f2354fbca3486078415b96e42d8f4305db0c998c0a934500a6bfe8e3b3b6"}
Oct 30 22:54:42.666092 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:42.666006    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf" event={"ID":"bb586958-b1df-4396-8ca4-f2bbf71addbc","Type":"ContainerDied","Data":"08c9afe2ca71df65dd7eabfb1d8934b20a694bb6914a590768fa2eed03ca3b8c"}
Oct 30 22:54:42.666092 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:42.666032    4871 scope.go:122] "RemoveContainer" containerID="4d08f2354fbca3486078415b96e42d8f4305db0c998c0a934500a6bfe8e3b3b6"
Oct 30 22:54:42.678191 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:42.678172    4871 scope.go:122] "RemoveContainer" containerID="4d08f2354fbca3486078415b96e42d8f4305db0c998c0a934500a6bfe8e3b3b6"
Oct 30 22:54:42.678516 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:54:42.678494    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d08f2354fbca3486078415b96e42d8f4305db0c998c0a934500a6bfe8e3b3b6\": container with ID starting with 4d08f2354fbca3486078415b96e42d8f4305db0c998c0a934500a6bfe8e3b3b6 not found: ID does not exist" containerID="4d08f2354fbca3486078415b96e42d8f4305db0c998c0a934500a6bfe8e3b3b6"
Oct 30 22:54:42.678580 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:42.678526    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d08f2354fbca3486078415b96e42d8f4305db0c998c0a934500a6bfe8e3b3b6"} err="failed to get container status \"4d08f2354fbca3486078415b96e42d8f4305db0c998c0a934500a6bfe8e3b3b6\": rpc error: code = NotFound desc = could not find container \"4d08f2354fbca3486078415b96e42d8f4305db0c998c0a934500a6bfe8e3b3b6\": container with ID starting with 4d08f2354fbca3486078415b96e42d8f4305db0c998c0a934500a6bfe8e3b3b6 not found: ID does not exist"
Oct 30 22:54:42.840499 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:42.840467    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf"]
Oct 30 22:54:42.895006 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:42.894966    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-9af9e7-5d59546d5b-mwwjf"]
Oct 30 22:54:43.343577 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:43.343541    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:43.510285 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:43.510232    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:43.663000 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:43.662965    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:43.959407 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:43.959325    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bb586958-b1df-4396-8ca4-f2bbf71addbc" path="/var/lib/kubelet/pods/bb586958-b1df-4396-8ca4-f2bbf71addbc/volumes"
Oct 30 22:54:44.218943 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:44.218853    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:48.209316 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:48.209213    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:50.750555 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:50.750522    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:51.023891 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:51.023816    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:51.287020 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:51.286917    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:52.934606 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:52.934568    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:55.529173 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:55.529139    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:56.964492 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:56.964451    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:57.823959 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:54:57.823929    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:02.403902 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:02.403864    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:03.957470 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:03.957425    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:04.116834 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:04.116793    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:05.567795 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:55:05.567758    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:55:06.895308 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:06.895274    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:11.502422 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:11.502384    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:16.121798 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:16.121747    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:20.102985 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:20.102948    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj"]
Oct 30 22:55:20.103504 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:20.103160    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj" podUID="f2cc9fd6-d809-460e-a2dc-dc6ea02d3592" containerName="maas-api" containerID="cri-o://13bb5f870006cbc04f4b776b20d76b208a5e36b609358580955ce416c2d321e7" gracePeriod=30
Oct 30 22:55:20.164345 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:20.164307    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:20.332020 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:20.331982    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:20.380444 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:20.380408    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj"
Oct 30 22:55:20.414003 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:20.413970    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f2cc9fd6-d809-460e-a2dc-dc6ea02d3592-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2cc9fd6-d809-460e-a2dc-dc6ea02d3592-openshift-service-ca\") pod \"f2cc9fd6-d809-460e-a2dc-dc6ea02d3592\" (UID: \"f2cc9fd6-d809-460e-a2dc-dc6ea02d3592\") "
Oct 30 22:55:20.414228 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:20.414110    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f2cc9fd6-d809-460e-a2dc-dc6ea02d3592-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2cc9fd6-d809-460e-a2dc-dc6ea02d3592-metrics-certs\") pod \"f2cc9fd6-d809-460e-a2dc-dc6ea02d3592\" (UID: \"f2cc9fd6-d809-460e-a2dc-dc6ea02d3592\") "
Oct 30 22:55:20.414228 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:20.414162    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f2cc9fd6-d809-460e-a2dc-dc6ea02d3592-kube-api-access-89hxn\" (UniqueName: \"kubernetes.io/projected/f2cc9fd6-d809-460e-a2dc-dc6ea02d3592-kube-api-access-89hxn\") pod \"f2cc9fd6-d809-460e-a2dc-dc6ea02d3592\" (UID: \"f2cc9fd6-d809-460e-a2dc-dc6ea02d3592\") "
Oct 30 22:55:20.414228 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:20.414209    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f2cc9fd6-d809-460e-a2dc-dc6ea02d3592-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2cc9fd6-d809-460e-a2dc-dc6ea02d3592-maas-api-tls\") pod \"f2cc9fd6-d809-460e-a2dc-dc6ea02d3592\" (UID: \"f2cc9fd6-d809-460e-a2dc-dc6ea02d3592\") "
Oct 30 22:55:20.414429 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:20.414343    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2cc9fd6-d809-460e-a2dc-dc6ea02d3592-openshift-service-ca" pod "f2cc9fd6-d809-460e-a2dc-dc6ea02d3592" (UID: "f2cc9fd6-d809-460e-a2dc-dc6ea02d3592"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:55:20.414592 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:20.414573    4871 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2cc9fd6-d809-460e-a2dc-dc6ea02d3592-openshift-service-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:20.416428 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:20.416390    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2cc9fd6-d809-460e-a2dc-dc6ea02d3592-kube-api-access-89hxn" pod "f2cc9fd6-d809-460e-a2dc-dc6ea02d3592" (UID: "f2cc9fd6-d809-460e-a2dc-dc6ea02d3592"). InnerVolumeSpecName "kube-api-access-89hxn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:55:20.416428 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:20.416413    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2cc9fd6-d809-460e-a2dc-dc6ea02d3592-maas-api-tls" pod "f2cc9fd6-d809-460e-a2dc-dc6ea02d3592" (UID: "f2cc9fd6-d809-460e-a2dc-dc6ea02d3592"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:55:20.416585 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:20.416433    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2cc9fd6-d809-460e-a2dc-dc6ea02d3592-metrics-certs" pod "f2cc9fd6-d809-460e-a2dc-dc6ea02d3592" (UID: "f2cc9fd6-d809-460e-a2dc-dc6ea02d3592"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:55:20.515767 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:20.515731    4871 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2cc9fd6-d809-460e-a2dc-dc6ea02d3592-metrics-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:20.515767 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:20.515761    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-89hxn\" (UniqueName: \"kubernetes.io/projected/f2cc9fd6-d809-460e-a2dc-dc6ea02d3592-kube-api-access-89hxn\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:20.515767 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:20.515773    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2cc9fd6-d809-460e-a2dc-dc6ea02d3592-maas-api-tls\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:20.898223 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:20.898189    4871 generic.go:356] "Generic (PLEG): container finished" podID="f2cc9fd6-d809-460e-a2dc-dc6ea02d3592" containerID="13bb5f870006cbc04f4b776b20d76b208a5e36b609358580955ce416c2d321e7" exitCode=0
Oct 30 22:55:20.898458 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:20.898251    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj"
Oct 30 22:55:20.898458 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:20.898268    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj" event={"ID":"f2cc9fd6-d809-460e-a2dc-dc6ea02d3592","Type":"ContainerDied","Data":"13bb5f870006cbc04f4b776b20d76b208a5e36b609358580955ce416c2d321e7"}
Oct 30 22:55:20.898458 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:20.898330    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj" event={"ID":"f2cc9fd6-d809-460e-a2dc-dc6ea02d3592","Type":"ContainerDied","Data":"00162065a904e8d6c654f222579946e94ff603415be43e9767b9f59bab80c05e"}
Oct 30 22:55:20.898458 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:20.898359    4871 scope.go:122] "RemoveContainer" containerID="13bb5f870006cbc04f4b776b20d76b208a5e36b609358580955ce416c2d321e7"
Oct 30 22:55:20.909390 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:20.909367    4871 scope.go:122] "RemoveContainer" containerID="13bb5f870006cbc04f4b776b20d76b208a5e36b609358580955ce416c2d321e7"
Oct 30 22:55:20.909676 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:55:20.909655    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13bb5f870006cbc04f4b776b20d76b208a5e36b609358580955ce416c2d321e7\": container with ID starting with 13bb5f870006cbc04f4b776b20d76b208a5e36b609358580955ce416c2d321e7 not found: ID does not exist" containerID="13bb5f870006cbc04f4b776b20d76b208a5e36b609358580955ce416c2d321e7"
Oct 30 22:55:20.909726 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:20.909685    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13bb5f870006cbc04f4b776b20d76b208a5e36b609358580955ce416c2d321e7"} err="failed to get container status \"13bb5f870006cbc04f4b776b20d76b208a5e36b609358580955ce416c2d321e7\": rpc error: code = NotFound desc = could not find container \"13bb5f870006cbc04f4b776b20d76b208a5e36b609358580955ce416c2d321e7\": container with ID starting with 13bb5f870006cbc04f4b776b20d76b208a5e36b609358580955ce416c2d321e7 not found: ID does not exist"
Oct 30 22:55:21.079128 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:21.079094    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj"]
Oct 30 22:55:21.081796 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:21.081760    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-6fc60c-5f75579588-hfgvj"]
Oct 30 22:55:21.691438 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:21.691402    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:21.935917 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:21.935864    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:21.962220 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:21.962139    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f2cc9fd6-d809-460e-a2dc-dc6ea02d3592" path="/var/lib/kubelet/pods/f2cc9fd6-d809-460e-a2dc-dc6ea02d3592/volumes"
Oct 30 22:55:22.111453 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:22.111397    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:22.608599 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:22.608553    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:26.825442 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:26.825387    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:28.750523 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:28.750477    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:28.981183 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:28.981152    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:29.132588 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:29.132553    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"]
Oct 30 22:55:29.132796 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:29.132771    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2" podUID="478b54f8-6c41-42ad-ac53-cb6144f07773" containerName="istio-proxy" containerID="cri-o://e34feda1f78b9316752feaba026dbdb622b99d59b3912538139b14ad5cc00a92" gracePeriod=30
Oct 30 22:55:29.146961 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:29.146901    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:29.950130 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:29.950093    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-c29159-pn6qc"]
Oct 30 22:55:29.950554 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:29.950539    4871 cpu_manager.go:414] "containerMap: removing container" podUID="bb586958-b1df-4396-8ca4-f2bbf71addbc" containerName="maas-api"
Oct 30 22:55:29.950554 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:29.950554    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bb586958-b1df-4396-8ca4-f2bbf71addbc" containerName="maas-api"
Oct 30 22:55:29.950629 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:29.950563    4871 cpu_manager.go:414] "containerMap: removing container" podUID="f2cc9fd6-d809-460e-a2dc-dc6ea02d3592" containerName="maas-api"
Oct 30 22:55:29.950629 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:29.950568    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f2cc9fd6-d809-460e-a2dc-dc6ea02d3592" containerName="maas-api"
Oct 30 22:55:29.950629 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:29.950587    4871 cpu_manager.go:414] "containerMap: removing container" podUID="406a92fa-f298-4c04-9cde-4bdc43c9c385" containerName="payload-processing"
Oct 30 22:55:29.950629 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:29.950592    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="406a92fa-f298-4c04-9cde-4bdc43c9c385" containerName="payload-processing"
Oct 30 22:55:29.950629 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:29.950599    4871 cpu_manager.go:414] "containerMap: removing container" podUID="2a1963fe-96ab-4a79-a186-8d7aee904fb2" containerName="payload-processing"
Oct 30 22:55:29.950629 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:29.950603    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2a1963fe-96ab-4a79-a186-8d7aee904fb2" containerName="payload-processing"
Oct 30 22:55:29.950629 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:29.950618    4871 cpu_manager.go:414] "containerMap: removing container" podUID="cb52525e-ca08-4be0-afc0-954385b1b723" containerName="payload-processing"
Oct 30 22:55:29.950629 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:29.950623    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cb52525e-ca08-4be0-afc0-954385b1b723" containerName="payload-processing"
Oct 30 22:55:29.950887 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:29.950687    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="bb586958-b1df-4396-8ca4-f2bbf71addbc" containerName="maas-api"
Oct 30 22:55:29.950887 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:29.950695    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="f2cc9fd6-d809-460e-a2dc-dc6ea02d3592" containerName="maas-api"
Oct 30 22:55:29.950887 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:29.950703    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="406a92fa-f298-4c04-9cde-4bdc43c9c385" containerName="payload-processing"
Oct 30 22:55:29.950887 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:29.950711    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="2a1963fe-96ab-4a79-a186-8d7aee904fb2" containerName="payload-processing"
Oct 30 22:55:29.950887 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:29.950720    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="cb52525e-ca08-4be0-afc0-954385b1b723" containerName="payload-processing"
Oct 30 22:55:29.955105 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:29.955082    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-c29159-pn6qc"
Oct 30 22:55:30.008289 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:30.008252    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-c29159-pn6qc"]
Oct 30 22:55:30.015311 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:30.015280    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a076d875-0917-48f2-a6a9-15338a48e254-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w0-c29159-pn6qc\" (UID: \"a076d875-0917-48f2-a6a9-15338a48e254\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-c29159-pn6qc"
Oct 30 22:55:30.015470 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:30.015324    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a076d875-0917-48f2-a6a9-15338a48e254-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w0-c29159-pn6qc\" (UID: \"a076d875-0917-48f2-a6a9-15338a48e254\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-c29159-pn6qc"
Oct 30 22:55:30.015541 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:30.015474    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqpvj\" (UniqueName: \"kubernetes.io/projected/a076d875-0917-48f2-a6a9-15338a48e254-kube-api-access-nqpvj\") pod \"maas-api-revoke-keys-e2e-shared-b-w0-c29159-pn6qc\" (UID: \"a076d875-0917-48f2-a6a9-15338a48e254\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-c29159-pn6qc"
Oct 30 22:55:30.057784 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:30.057739    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:55:30.057784 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:30.057776    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:55:30.057998 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:30.057785    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:55:30.057998 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:30.057827    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mlvtf\""
Oct 30 22:55:30.117061 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:30.117019    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a076d875-0917-48f2-a6a9-15338a48e254-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w0-c29159-pn6qc\" (UID: \"a076d875-0917-48f2-a6a9-15338a48e254\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-c29159-pn6qc"
Oct 30 22:55:30.117061 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:30.117073    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a076d875-0917-48f2-a6a9-15338a48e254-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w0-c29159-pn6qc\" (UID: \"a076d875-0917-48f2-a6a9-15338a48e254\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-c29159-pn6qc"
Oct 30 22:55:30.117386 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:30.117265    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqpvj\" (UniqueName: \"kubernetes.io/projected/a076d875-0917-48f2-a6a9-15338a48e254-kube-api-access-nqpvj\") pod \"maas-api-revoke-keys-e2e-shared-b-w0-c29159-pn6qc\" (UID: \"a076d875-0917-48f2-a6a9-15338a48e254\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-c29159-pn6qc"
Oct 30 22:55:30.117850 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:30.117829    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a076d875-0917-48f2-a6a9-15338a48e254-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w0-c29159-pn6qc\" (UID: \"a076d875-0917-48f2-a6a9-15338a48e254\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-c29159-pn6qc"
Oct 30 22:55:30.119660 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:30.119640    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a076d875-0917-48f2-a6a9-15338a48e254-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w0-c29159-pn6qc\" (UID: \"a076d875-0917-48f2-a6a9-15338a48e254\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-c29159-pn6qc"
Oct 30 22:55:30.181109 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:30.181078    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqpvj\" (UniqueName: \"kubernetes.io/projected/a076d875-0917-48f2-a6a9-15338a48e254-kube-api-access-nqpvj\") pod \"maas-api-revoke-keys-e2e-shared-b-w0-c29159-pn6qc\" (UID: \"a076d875-0917-48f2-a6a9-15338a48e254\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-c29159-pn6qc"
Oct 30 22:55:30.269208 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:30.269119    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-c29159-pn6qc"
Oct 30 22:55:30.642786 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:55:30.642741    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda076d875_0917_48f2_a6a9_15338a48e254.slice/crio-7a1998724ea7a51f6fecf541049398b77586570adb41301e3d46cd1f56babf71 WatchSource:0}: Error finding container 7a1998724ea7a51f6fecf541049398b77586570adb41301e3d46cd1f56babf71: Status 404 returned error can't find the container with id 7a1998724ea7a51f6fecf541049398b77586570adb41301e3d46cd1f56babf71
Oct 30 22:55:30.646957 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:30.646766    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-c29159-pn6qc"]
Oct 30 22:55:30.967415 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:30.967376    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-c29159-pn6qc" event={"ID":"a076d875-0917-48f2-a6a9-15338a48e254","Type":"ContainerStarted","Data":"10d0306f0152b3d602c6d34b2630ca19265126fe7767318d71322a6dd045f2d0"}
Oct 30 22:55:30.967828 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:30.967435    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-c29159-pn6qc" event={"ID":"a076d875-0917-48f2-a6a9-15338a48e254","Type":"ContainerStarted","Data":"7a1998724ea7a51f6fecf541049398b77586570adb41301e3d46cd1f56babf71"}
Oct 30 22:55:31.090041 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:31.089986    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-c29159-pn6qc" podStartSLOduration=2.089970449 podStartE2EDuration="2.089970449s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:55:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:55:31.087806079 +0000 UTC m=+4673.265294708" watchObservedRunningTime="2026-10-30 22:55:31.089970449 +0000 UTC m=+4673.267459063"
Oct 30 22:55:31.644579 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:31.644530    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:31.974498 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:31.974463    4871 generic.go:356] "Generic (PLEG): container finished" podID="a076d875-0917-48f2-a6a9-15338a48e254" containerID="10d0306f0152b3d602c6d34b2630ca19265126fe7767318d71322a6dd045f2d0" exitCode=0
Oct 30 22:55:31.974911 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:31.974511    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-c29159-pn6qc" event={"ID":"a076d875-0917-48f2-a6a9-15338a48e254","Type":"ContainerDied","Data":"10d0306f0152b3d602c6d34b2630ca19265126fe7767318d71322a6dd045f2d0"}
Oct 30 22:55:33.134273 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:33.134209    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-c29159-pn6qc"
Oct 30 22:55:33.252656 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:33.252554    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a076d875-0917-48f2-a6a9-15338a48e254-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a076d875-0917-48f2-a6a9-15338a48e254-maas-api-openshift-ca\") pod \"a076d875-0917-48f2-a6a9-15338a48e254\" (UID: \"a076d875-0917-48f2-a6a9-15338a48e254\") "
Oct 30 22:55:33.252656 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:33.252634    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a076d875-0917-48f2-a6a9-15338a48e254-kube-api-access-nqpvj\" (UniqueName: \"kubernetes.io/projected/a076d875-0917-48f2-a6a9-15338a48e254-kube-api-access-nqpvj\") pod \"a076d875-0917-48f2-a6a9-15338a48e254\" (UID: \"a076d875-0917-48f2-a6a9-15338a48e254\") "
Oct 30 22:55:33.253716 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:33.252816    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a076d875-0917-48f2-a6a9-15338a48e254-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a076d875-0917-48f2-a6a9-15338a48e254-maas-api-xks-ca\") pod \"a076d875-0917-48f2-a6a9-15338a48e254\" (UID: \"a076d875-0917-48f2-a6a9-15338a48e254\") "
Oct 30 22:55:33.253716 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:33.253451    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a076d875-0917-48f2-a6a9-15338a48e254-maas-api-openshift-ca" pod "a076d875-0917-48f2-a6a9-15338a48e254" (UID: "a076d875-0917-48f2-a6a9-15338a48e254"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:55:33.255582 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:33.255540    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a076d875-0917-48f2-a6a9-15338a48e254-kube-api-access-nqpvj" pod "a076d875-0917-48f2-a6a9-15338a48e254" (UID: "a076d875-0917-48f2-a6a9-15338a48e254"). InnerVolumeSpecName "kube-api-access-nqpvj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:55:33.255692 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:33.255599    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a076d875-0917-48f2-a6a9-15338a48e254-maas-api-xks-ca" pod "a076d875-0917-48f2-a6a9-15338a48e254" (UID: "a076d875-0917-48f2-a6a9-15338a48e254"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:55:33.354021 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:33.353982    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a076d875-0917-48f2-a6a9-15338a48e254-maas-api-xks-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:33.354021 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:33.354016    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a076d875-0917-48f2-a6a9-15338a48e254-maas-api-openshift-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:33.354021 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:33.354026    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nqpvj\" (UniqueName: \"kubernetes.io/projected/a076d875-0917-48f2-a6a9-15338a48e254-kube-api-access-nqpvj\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:33.989314 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:33.989283    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-c29159-pn6qc"
Oct 30 22:55:33.989314 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:33.989305    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-c29159-pn6qc" event={"ID":"a076d875-0917-48f2-a6a9-15338a48e254","Type":"ContainerDied","Data":"7a1998724ea7a51f6fecf541049398b77586570adb41301e3d46cd1f56babf71"}
Oct 30 22:55:33.989651 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:33.989354    4871 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7a1998724ea7a51f6fecf541049398b77586570adb41301e3d46cd1f56babf71"
Oct 30 22:55:34.388637 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:34.388611    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"
Oct 30 22:55:34.461937 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:34.461901    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/478b54f8-6c41-42ad-ac53-cb6144f07773-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/478b54f8-6c41-42ad-ac53-cb6144f07773-workload-certs\") pod \"478b54f8-6c41-42ad-ac53-cb6144f07773\" (UID: \"478b54f8-6c41-42ad-ac53-cb6144f07773\") "
Oct 30 22:55:34.462119 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:34.461958    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/478b54f8-6c41-42ad-ac53-cb6144f07773-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/478b54f8-6c41-42ad-ac53-cb6144f07773-credential-socket\") pod \"478b54f8-6c41-42ad-ac53-cb6144f07773\" (UID: \"478b54f8-6c41-42ad-ac53-cb6144f07773\") "
Oct 30 22:55:34.462119 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:34.461988    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/478b54f8-6c41-42ad-ac53-cb6144f07773-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/478b54f8-6c41-42ad-ac53-cb6144f07773-istio-podinfo\") pod \"478b54f8-6c41-42ad-ac53-cb6144f07773\" (UID: \"478b54f8-6c41-42ad-ac53-cb6144f07773\") "
Oct 30 22:55:34.462119 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:34.462041    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/478b54f8-6c41-42ad-ac53-cb6144f07773-istio-token\" (UniqueName: \"kubernetes.io/projected/478b54f8-6c41-42ad-ac53-cb6144f07773-istio-token\") pod \"478b54f8-6c41-42ad-ac53-cb6144f07773\" (UID: \"478b54f8-6c41-42ad-ac53-cb6144f07773\") "
Oct 30 22:55:34.462119 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:34.462101    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/478b54f8-6c41-42ad-ac53-cb6144f07773-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/478b54f8-6c41-42ad-ac53-cb6144f07773-istio-data\") pod \"478b54f8-6c41-42ad-ac53-cb6144f07773\" (UID: \"478b54f8-6c41-42ad-ac53-cb6144f07773\") "
Oct 30 22:55:34.462414 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:34.462123    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/478b54f8-6c41-42ad-ac53-cb6144f07773-kube-api-access-95d95\" (UniqueName: \"kubernetes.io/projected/478b54f8-6c41-42ad-ac53-cb6144f07773-kube-api-access-95d95\") pod \"478b54f8-6c41-42ad-ac53-cb6144f07773\" (UID: \"478b54f8-6c41-42ad-ac53-cb6144f07773\") "
Oct 30 22:55:34.462414 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:34.462150    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/478b54f8-6c41-42ad-ac53-cb6144f07773-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/478b54f8-6c41-42ad-ac53-cb6144f07773-workload-socket\") pod \"478b54f8-6c41-42ad-ac53-cb6144f07773\" (UID: \"478b54f8-6c41-42ad-ac53-cb6144f07773\") "
Oct 30 22:55:34.462414 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:34.462189    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/478b54f8-6c41-42ad-ac53-cb6144f07773-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/478b54f8-6c41-42ad-ac53-cb6144f07773-istiod-ca-cert\") pod \"478b54f8-6c41-42ad-ac53-cb6144f07773\" (UID: \"478b54f8-6c41-42ad-ac53-cb6144f07773\") "
Oct 30 22:55:34.462414 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:34.462280    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/478b54f8-6c41-42ad-ac53-cb6144f07773-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/478b54f8-6c41-42ad-ac53-cb6144f07773-istio-envoy\") pod \"478b54f8-6c41-42ad-ac53-cb6144f07773\" (UID: \"478b54f8-6c41-42ad-ac53-cb6144f07773\") "
Oct 30 22:55:34.462414 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:34.462284    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/478b54f8-6c41-42ad-ac53-cb6144f07773-workload-certs" pod "478b54f8-6c41-42ad-ac53-cb6144f07773" (UID: "478b54f8-6c41-42ad-ac53-cb6144f07773"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:55:34.462414 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:34.462403    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/478b54f8-6c41-42ad-ac53-cb6144f07773-credential-socket" pod "478b54f8-6c41-42ad-ac53-cb6144f07773" (UID: "478b54f8-6c41-42ad-ac53-cb6144f07773"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:55:34.462755 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:34.462555    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/478b54f8-6c41-42ad-ac53-cb6144f07773-workload-socket" pod "478b54f8-6c41-42ad-ac53-cb6144f07773" (UID: "478b54f8-6c41-42ad-ac53-cb6144f07773"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:55:34.462755 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:34.462675    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/478b54f8-6c41-42ad-ac53-cb6144f07773-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:34.462755 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:34.462698    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/478b54f8-6c41-42ad-ac53-cb6144f07773-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:34.462755 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:34.462714    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/478b54f8-6c41-42ad-ac53-cb6144f07773-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:34.462954 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:34.462779    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/478b54f8-6c41-42ad-ac53-cb6144f07773-istio-data" pod "478b54f8-6c41-42ad-ac53-cb6144f07773" (UID: "478b54f8-6c41-42ad-ac53-cb6144f07773"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:55:34.462954 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:34.462845    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/478b54f8-6c41-42ad-ac53-cb6144f07773-istiod-ca-cert" pod "478b54f8-6c41-42ad-ac53-cb6144f07773" (UID: "478b54f8-6c41-42ad-ac53-cb6144f07773"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:55:34.464618 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:34.464591    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/478b54f8-6c41-42ad-ac53-cb6144f07773-istio-envoy" pod "478b54f8-6c41-42ad-ac53-cb6144f07773" (UID: "478b54f8-6c41-42ad-ac53-cb6144f07773"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:55:34.464773 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:34.464754    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/478b54f8-6c41-42ad-ac53-cb6144f07773-istio-podinfo" pod "478b54f8-6c41-42ad-ac53-cb6144f07773" (UID: "478b54f8-6c41-42ad-ac53-cb6144f07773"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:55:34.464874 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:34.464851    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/478b54f8-6c41-42ad-ac53-cb6144f07773-istio-token" pod "478b54f8-6c41-42ad-ac53-cb6144f07773" (UID: "478b54f8-6c41-42ad-ac53-cb6144f07773"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:55:34.464958 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:34.464940    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/478b54f8-6c41-42ad-ac53-cb6144f07773-kube-api-access-95d95" pod "478b54f8-6c41-42ad-ac53-cb6144f07773" (UID: "478b54f8-6c41-42ad-ac53-cb6144f07773"). InnerVolumeSpecName "kube-api-access-95d95". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:55:34.565475 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:34.564117    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/478b54f8-6c41-42ad-ac53-cb6144f07773-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:34.565475 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:34.564155    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-95d95\" (UniqueName: \"kubernetes.io/projected/478b54f8-6c41-42ad-ac53-cb6144f07773-kube-api-access-95d95\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:34.565475 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:34.564170    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/478b54f8-6c41-42ad-ac53-cb6144f07773-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:34.565475 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:34.564183    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/478b54f8-6c41-42ad-ac53-cb6144f07773-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:34.565475 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:34.564196    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/478b54f8-6c41-42ad-ac53-cb6144f07773-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:34.565475 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:34.564208    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/478b54f8-6c41-42ad-ac53-cb6144f07773-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:34.998486 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:34.998441    4871 generic.go:356] "Generic (PLEG): container finished" podID="478b54f8-6c41-42ad-ac53-cb6144f07773" containerID="e34feda1f78b9316752feaba026dbdb622b99d59b3912538139b14ad5cc00a92" exitCode=0
Oct 30 22:55:34.998681 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:34.998504    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"
Oct 30 22:55:34.998681 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:34.998534    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2" event={"ID":"478b54f8-6c41-42ad-ac53-cb6144f07773","Type":"ContainerDied","Data":"e34feda1f78b9316752feaba026dbdb622b99d59b3912538139b14ad5cc00a92"}
Oct 30 22:55:34.998681 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:34.998603    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2" event={"ID":"478b54f8-6c41-42ad-ac53-cb6144f07773","Type":"ContainerDied","Data":"aed50fe150b503675954880fbe3f93463d10582d970d4022478542ec7833596f"}
Oct 30 22:55:34.998681 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:34.998633    4871 scope.go:122] "RemoveContainer" containerID="e34feda1f78b9316752feaba026dbdb622b99d59b3912538139b14ad5cc00a92"
Oct 30 22:55:35.009815 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:35.009788    4871 scope.go:122] "RemoveContainer" containerID="f6f414d401e6b5799505fff1643916daf0a5f8c0be737f58c1484a5a7b96a6bc"
Oct 30 22:55:35.019419 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:35.019395    4871 scope.go:122] "RemoveContainer" containerID="e34feda1f78b9316752feaba026dbdb622b99d59b3912538139b14ad5cc00a92"
Oct 30 22:55:35.019756 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:55:35.019736    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e34feda1f78b9316752feaba026dbdb622b99d59b3912538139b14ad5cc00a92\": container with ID starting with e34feda1f78b9316752feaba026dbdb622b99d59b3912538139b14ad5cc00a92 not found: ID does not exist" containerID="e34feda1f78b9316752feaba026dbdb622b99d59b3912538139b14ad5cc00a92"
Oct 30 22:55:35.019804 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:35.019767    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e34feda1f78b9316752feaba026dbdb622b99d59b3912538139b14ad5cc00a92"} err="failed to get container status \"e34feda1f78b9316752feaba026dbdb622b99d59b3912538139b14ad5cc00a92\": rpc error: code = NotFound desc = could not find container \"e34feda1f78b9316752feaba026dbdb622b99d59b3912538139b14ad5cc00a92\": container with ID starting with e34feda1f78b9316752feaba026dbdb622b99d59b3912538139b14ad5cc00a92 not found: ID does not exist"
Oct 30 22:55:35.019804 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:35.019790    4871 scope.go:122] "RemoveContainer" containerID="f6f414d401e6b5799505fff1643916daf0a5f8c0be737f58c1484a5a7b96a6bc"
Oct 30 22:55:35.020033 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:55:35.020010    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6f414d401e6b5799505fff1643916daf0a5f8c0be737f58c1484a5a7b96a6bc\": container with ID starting with f6f414d401e6b5799505fff1643916daf0a5f8c0be737f58c1484a5a7b96a6bc not found: ID does not exist" containerID="f6f414d401e6b5799505fff1643916daf0a5f8c0be737f58c1484a5a7b96a6bc"
Oct 30 22:55:35.020080 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:35.020040    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6f414d401e6b5799505fff1643916daf0a5f8c0be737f58c1484a5a7b96a6bc"} err="failed to get container status \"f6f414d401e6b5799505fff1643916daf0a5f8c0be737f58c1484a5a7b96a6bc\": rpc error: code = NotFound desc = could not find container \"f6f414d401e6b5799505fff1643916daf0a5f8c0be737f58c1484a5a7b96a6bc\": container with ID starting with f6f414d401e6b5799505fff1643916daf0a5f8c0be737f58c1484a5a7b96a6bc not found: ID does not exist"
Oct 30 22:55:35.175858 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:35.175819    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"]
Oct 30 22:55:35.228377 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:35.228338    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-6fc60c-openshift-default-66fcd9d47f-4znq2"]
Oct 30 22:55:35.567847 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:55:35.567808    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:55:35.961005 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:35.960966    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="478b54f8-6c41-42ad-ac53-cb6144f07773" path="/var/lib/kubelet/pods/478b54f8-6c41-42ad-ac53-cb6144f07773/volumes"
Oct 30 22:55:36.607105 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:36.607073    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:40.586821 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:40.586778    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:41.169152 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:41.169093    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:41.349902 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:41.349870    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:42.427003 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:42.426969    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:45.588373 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:45.588340    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:45.856406 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:45.856319    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:46.054916 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:46.054884    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-c29159-pn6qc"]
Oct 30 22:55:46.107322 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:46.107219    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-c29159-pn6qc"]
Oct 30 22:55:46.515418 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:46.515389    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:47.269359 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:47.269324    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:47.959290 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:47.959257    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a076d875-0917-48f2-a6a9-15338a48e254" path="/var/lib/kubelet/pods/a076d875-0917-48f2-a6a9-15338a48e254/volumes"
Oct 30 22:55:52.002475 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:52.002418    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:53.036362 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:53.036325    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:53.210536 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:53.210501    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:53.308040 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:53.307947    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"]
Oct 30 22:55:53.308273 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:53.308226    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5" podUID="8acc9cc7-ebae-4d92-8e8c-c8ed8e615030" containerName="istio-proxy" containerID="cri-o://b892b4080ca61e0a9ef2d82e931206ba5e7fb8c28a9bce6d723a67f506ff93d2" gracePeriod=30
Oct 30 22:55:53.411422 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:53.411381    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:53.970810 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:53.970772    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:54.353380 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:54.353288    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-9a5623-c7cdg"]
Oct 30 22:55:54.353879 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:54.353728    4871 cpu_manager.go:414] "containerMap: removing container" podUID="478b54f8-6c41-42ad-ac53-cb6144f07773" containerName="istio-proxy"
Oct 30 22:55:54.353879 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:54.353740    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="478b54f8-6c41-42ad-ac53-cb6144f07773" containerName="istio-proxy"
Oct 30 22:55:54.353879 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:54.353754    4871 cpu_manager.go:414] "containerMap: removing container" podUID="a076d875-0917-48f2-a6a9-15338a48e254" containerName="revoke-keys"
Oct 30 22:55:54.353879 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:54.353759    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a076d875-0917-48f2-a6a9-15338a48e254" containerName="revoke-keys"
Oct 30 22:55:54.353879 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:54.353772    4871 cpu_manager.go:414] "containerMap: removing container" podUID="478b54f8-6c41-42ad-ac53-cb6144f07773" containerName="istio-proxy"
Oct 30 22:55:54.353879 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:54.353777    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="478b54f8-6c41-42ad-ac53-cb6144f07773" containerName="istio-proxy"
Oct 30 22:55:54.353879 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:54.353866    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="478b54f8-6c41-42ad-ac53-cb6144f07773" containerName="istio-proxy"
Oct 30 22:55:54.353879 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:54.353874    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="a076d875-0917-48f2-a6a9-15338a48e254" containerName="revoke-keys"
Oct 30 22:55:54.353879 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:54.353882    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="478b54f8-6c41-42ad-ac53-cb6144f07773" containerName="istio-proxy"
Oct 30 22:55:54.357055 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:54.357036    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-9a5623-c7cdg"
Oct 30 22:55:54.413105 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:54.413069    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-9a5623-c7cdg"]
Oct 30 22:55:54.461558 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:54.461520    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a92f92e4-ff49-47c9-9333-beb5ee94a46a-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-9a5623-c7cdg\" (UID: \"a92f92e4-ff49-47c9-9333-beb5ee94a46a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-9a5623-c7cdg"
Oct 30 22:55:54.461758 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:54.461737    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qz4x5\" (UniqueName: \"kubernetes.io/projected/a92f92e4-ff49-47c9-9333-beb5ee94a46a-kube-api-access-qz4x5\") pod \"maas-api-revoke-keys-e2e-worker-w0-9a5623-c7cdg\" (UID: \"a92f92e4-ff49-47c9-9333-beb5ee94a46a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-9a5623-c7cdg"
Oct 30 22:55:54.461833 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:54.461769    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a92f92e4-ff49-47c9-9333-beb5ee94a46a-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-9a5623-c7cdg\" (UID: \"a92f92e4-ff49-47c9-9333-beb5ee94a46a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-9a5623-c7cdg"
Oct 30 22:55:54.462067 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:54.462055    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:55:54.462134 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:54.462113    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:55:54.462180 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:54.462138    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:55:54.462180 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:54.462165    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mlvtf\""
Oct 30 22:55:54.562963 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:54.562929    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a92f92e4-ff49-47c9-9333-beb5ee94a46a-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-9a5623-c7cdg\" (UID: \"a92f92e4-ff49-47c9-9333-beb5ee94a46a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-9a5623-c7cdg"
Oct 30 22:55:54.563162 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:54.563022    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qz4x5\" (UniqueName: \"kubernetes.io/projected/a92f92e4-ff49-47c9-9333-beb5ee94a46a-kube-api-access-qz4x5\") pod \"maas-api-revoke-keys-e2e-worker-w0-9a5623-c7cdg\" (UID: \"a92f92e4-ff49-47c9-9333-beb5ee94a46a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-9a5623-c7cdg"
Oct 30 22:55:54.563162 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:54.563042    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a92f92e4-ff49-47c9-9333-beb5ee94a46a-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-9a5623-c7cdg\" (UID: \"a92f92e4-ff49-47c9-9333-beb5ee94a46a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-9a5623-c7cdg"
Oct 30 22:55:54.563667 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:54.563644    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a92f92e4-ff49-47c9-9333-beb5ee94a46a-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-9a5623-c7cdg\" (UID: \"a92f92e4-ff49-47c9-9333-beb5ee94a46a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-9a5623-c7cdg"
Oct 30 22:55:54.565487 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:54.565455    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a92f92e4-ff49-47c9-9333-beb5ee94a46a-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-9a5623-c7cdg\" (UID: \"a92f92e4-ff49-47c9-9333-beb5ee94a46a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-9a5623-c7cdg"
Oct 30 22:55:54.626831 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:54.626796    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qz4x5\" (UniqueName: \"kubernetes.io/projected/a92f92e4-ff49-47c9-9333-beb5ee94a46a-kube-api-access-qz4x5\") pod \"maas-api-revoke-keys-e2e-worker-w0-9a5623-c7cdg\" (UID: \"a92f92e4-ff49-47c9-9333-beb5ee94a46a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-9a5623-c7cdg"
Oct 30 22:55:54.670040 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:54.670007    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-9a5623-c7cdg"
Oct 30 22:55:55.041882 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:55:55.041832    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda92f92e4_ff49_47c9_9333_beb5ee94a46a.slice/crio-e7234a743003621f151a6dcfa778d9225c69756c6800b7b3f8f05fd5bb9037e8 WatchSource:0}: Error finding container e7234a743003621f151a6dcfa778d9225c69756c6800b7b3f8f05fd5bb9037e8: Status 404 returned error can't find the container with id e7234a743003621f151a6dcfa778d9225c69756c6800b7b3f8f05fd5bb9037e8
Oct 30 22:55:55.044389 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:55.044368    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-9a5623-c7cdg"]
Oct 30 22:55:55.125892 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:55.125856    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-9a5623-c7cdg" event={"ID":"a92f92e4-ff49-47c9-9333-beb5ee94a46a","Type":"ContainerStarted","Data":"e7234a743003621f151a6dcfa778d9225c69756c6800b7b3f8f05fd5bb9037e8"}
Oct 30 22:55:55.852132 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:55.852095    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:56.134174 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:56.134132    4871 generic.go:356] "Generic (PLEG): container finished" podID="a92f92e4-ff49-47c9-9333-beb5ee94a46a" containerID="ab9699e145968044b932d27732a62336d9453a0683e417bb010fba4c00c745a5" exitCode=0
Oct 30 22:55:56.134351 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:56.134213    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-9a5623-c7cdg" event={"ID":"a92f92e4-ff49-47c9-9333-beb5ee94a46a","Type":"ContainerDied","Data":"ab9699e145968044b932d27732a62336d9453a0683e417bb010fba4c00c745a5"}
Oct 30 22:55:57.096199 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:57.096132    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:57.288493 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:57.288462    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-9a5623-c7cdg"
Oct 30 22:55:57.298778 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:57.298741    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:57.391866 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:57.391828    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a92f92e4-ff49-47c9-9333-beb5ee94a46a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a92f92e4-ff49-47c9-9333-beb5ee94a46a-maas-api-xks-ca\") pod \"a92f92e4-ff49-47c9-9333-beb5ee94a46a\" (UID: \"a92f92e4-ff49-47c9-9333-beb5ee94a46a\") "
Oct 30 22:55:57.392045 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:57.391885    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a92f92e4-ff49-47c9-9333-beb5ee94a46a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a92f92e4-ff49-47c9-9333-beb5ee94a46a-maas-api-openshift-ca\") pod \"a92f92e4-ff49-47c9-9333-beb5ee94a46a\" (UID: \"a92f92e4-ff49-47c9-9333-beb5ee94a46a\") "
Oct 30 22:55:57.392045 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:57.392003    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a92f92e4-ff49-47c9-9333-beb5ee94a46a-kube-api-access-qz4x5\" (UniqueName: \"kubernetes.io/projected/a92f92e4-ff49-47c9-9333-beb5ee94a46a-kube-api-access-qz4x5\") pod \"a92f92e4-ff49-47c9-9333-beb5ee94a46a\" (UID: \"a92f92e4-ff49-47c9-9333-beb5ee94a46a\") "
Oct 30 22:55:57.392363 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:57.392333    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a92f92e4-ff49-47c9-9333-beb5ee94a46a-maas-api-openshift-ca" pod "a92f92e4-ff49-47c9-9333-beb5ee94a46a" (UID: "a92f92e4-ff49-47c9-9333-beb5ee94a46a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:55:57.394162 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:57.394139    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a92f92e4-ff49-47c9-9333-beb5ee94a46a-kube-api-access-qz4x5" pod "a92f92e4-ff49-47c9-9333-beb5ee94a46a" (UID: "a92f92e4-ff49-47c9-9333-beb5ee94a46a"). InnerVolumeSpecName "kube-api-access-qz4x5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:55:57.394162 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:57.394138    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a92f92e4-ff49-47c9-9333-beb5ee94a46a-maas-api-xks-ca" pod "a92f92e4-ff49-47c9-9333-beb5ee94a46a" (UID: "a92f92e4-ff49-47c9-9333-beb5ee94a46a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:55:57.492716 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:57.492679    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qz4x5\" (UniqueName: \"kubernetes.io/projected/a92f92e4-ff49-47c9-9333-beb5ee94a46a-kube-api-access-qz4x5\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:57.492716 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:57.492714    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a92f92e4-ff49-47c9-9333-beb5ee94a46a-maas-api-xks-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:57.492930 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:57.492726    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a92f92e4-ff49-47c9-9333-beb5ee94a46a-maas-api-openshift-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:58.157498 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:58.157457    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-9a5623-c7cdg"
Oct 30 22:55:58.157918 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:58.157494    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-9a5623-c7cdg" event={"ID":"a92f92e4-ff49-47c9-9333-beb5ee94a46a","Type":"ContainerDied","Data":"e7234a743003621f151a6dcfa778d9225c69756c6800b7b3f8f05fd5bb9037e8"}
Oct 30 22:55:58.157918 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:58.157547    4871 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e7234a743003621f151a6dcfa778d9225c69756c6800b7b3f8f05fd5bb9037e8"
Oct 30 22:55:58.565907 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:58.565878    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"
Oct 30 22:55:58.604100 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:58.604015    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-istio-data\") pod \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\" (UID: \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\") "
Oct 30 22:55:58.604100 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:58.604061    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-istiod-ca-cert\") pod \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\" (UID: \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\") "
Oct 30 22:55:58.604394 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:58.604112    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-istio-envoy\") pod \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\" (UID: \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\") "
Oct 30 22:55:58.604394 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:58.604149    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-istio-token\" (UniqueName: \"kubernetes.io/projected/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-istio-token\") pod \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\" (UID: \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\") "
Oct 30 22:55:58.604394 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:58.604175    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-kube-api-access-hkh5j\" (UniqueName: \"kubernetes.io/projected/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-kube-api-access-hkh5j\") pod \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\" (UID: \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\") "
Oct 30 22:55:58.604394 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:58.604221    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-istio-podinfo\") pod \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\" (UID: \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\") "
Oct 30 22:55:58.604394 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:58.604289    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-workload-socket\") pod \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\" (UID: \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\") "
Oct 30 22:55:58.604394 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:58.604329    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-workload-certs\") pod \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\" (UID: \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\") "
Oct 30 22:55:58.604728 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:58.604404    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-istio-data" pod "8acc9cc7-ebae-4d92-8e8c-c8ed8e615030" (UID: "8acc9cc7-ebae-4d92-8e8c-c8ed8e615030"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:55:58.604728 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:58.604424    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-credential-socket\") pod \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\" (UID: \"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030\") "
Oct 30 22:55:58.604728 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:58.604673    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-workload-certs" pod "8acc9cc7-ebae-4d92-8e8c-c8ed8e615030" (UID: "8acc9cc7-ebae-4d92-8e8c-c8ed8e615030"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:55:58.604728 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:58.604672    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-istiod-ca-cert" pod "8acc9cc7-ebae-4d92-8e8c-c8ed8e615030" (UID: "8acc9cc7-ebae-4d92-8e8c-c8ed8e615030"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:55:58.604934 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:58.604855    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-workload-socket" pod "8acc9cc7-ebae-4d92-8e8c-c8ed8e615030" (UID: "8acc9cc7-ebae-4d92-8e8c-c8ed8e615030"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:55:58.604934 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:58.604867    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-credential-socket" pod "8acc9cc7-ebae-4d92-8e8c-c8ed8e615030" (UID: "8acc9cc7-ebae-4d92-8e8c-c8ed8e615030"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:55:58.605054 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:58.604990    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:58.605054 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:58.605003    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:58.605054 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:58.605013    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:58.606638 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:58.606608    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-istio-podinfo" pod "8acc9cc7-ebae-4d92-8e8c-c8ed8e615030" (UID: "8acc9cc7-ebae-4d92-8e8c-c8ed8e615030"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:55:58.606775 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:58.606641    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-istio-token" pod "8acc9cc7-ebae-4d92-8e8c-c8ed8e615030" (UID: "8acc9cc7-ebae-4d92-8e8c-c8ed8e615030"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:55:58.606893 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:58.606872    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-istio-envoy" pod "8acc9cc7-ebae-4d92-8e8c-c8ed8e615030" (UID: "8acc9cc7-ebae-4d92-8e8c-c8ed8e615030"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:55:58.606952 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:58.606889    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-kube-api-access-hkh5j" pod "8acc9cc7-ebae-4d92-8e8c-c8ed8e615030" (UID: "8acc9cc7-ebae-4d92-8e8c-c8ed8e615030"). InnerVolumeSpecName "kube-api-access-hkh5j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:55:58.706566 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:58.706524    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:58.706566 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:58.706565    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:58.706796 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:58.706583    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkh5j\" (UniqueName: \"kubernetes.io/projected/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-kube-api-access-hkh5j\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:58.706796 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:58.706599    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:58.706796 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:58.706613    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:58.706796 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:58.706628    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:58.916267 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:58.916206    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:59.164758 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:59.164722    4871 generic.go:356] "Generic (PLEG): container finished" podID="8acc9cc7-ebae-4d92-8e8c-c8ed8e615030" containerID="b892b4080ca61e0a9ef2d82e931206ba5e7fb8c28a9bce6d723a67f506ff93d2" exitCode=0
Oct 30 22:55:59.165296 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:59.164789    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"
Oct 30 22:55:59.165296 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:59.164812    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5" event={"ID":"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030","Type":"ContainerDied","Data":"b892b4080ca61e0a9ef2d82e931206ba5e7fb8c28a9bce6d723a67f506ff93d2"}
Oct 30 22:55:59.165296 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:59.164874    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5" event={"ID":"8acc9cc7-ebae-4d92-8e8c-c8ed8e615030","Type":"ContainerDied","Data":"55d6a8ebf9a2c233ce1633ea64d5fda9fd51c3bae3d354034ea5e816e37c881d"}
Oct 30 22:55:59.165296 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:59.164899    4871 scope.go:122] "RemoveContainer" containerID="b892b4080ca61e0a9ef2d82e931206ba5e7fb8c28a9bce6d723a67f506ff93d2"
Oct 30 22:55:59.176899 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:59.176877    4871 scope.go:122] "RemoveContainer" containerID="3214dd3b7a2df7d71e4ac79864cad8403ab0b9eb2556519fbd6f2d6fe52d17cd"
Oct 30 22:55:59.186300 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:59.186271    4871 scope.go:122] "RemoveContainer" containerID="b892b4080ca61e0a9ef2d82e931206ba5e7fb8c28a9bce6d723a67f506ff93d2"
Oct 30 22:55:59.186624 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:55:59.186597    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b892b4080ca61e0a9ef2d82e931206ba5e7fb8c28a9bce6d723a67f506ff93d2\": container with ID starting with b892b4080ca61e0a9ef2d82e931206ba5e7fb8c28a9bce6d723a67f506ff93d2 not found: ID does not exist" containerID="b892b4080ca61e0a9ef2d82e931206ba5e7fb8c28a9bce6d723a67f506ff93d2"
Oct 30 22:55:59.186710 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:59.186635    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b892b4080ca61e0a9ef2d82e931206ba5e7fb8c28a9bce6d723a67f506ff93d2"} err="failed to get container status \"b892b4080ca61e0a9ef2d82e931206ba5e7fb8c28a9bce6d723a67f506ff93d2\": rpc error: code = NotFound desc = could not find container \"b892b4080ca61e0a9ef2d82e931206ba5e7fb8c28a9bce6d723a67f506ff93d2\": container with ID starting with b892b4080ca61e0a9ef2d82e931206ba5e7fb8c28a9bce6d723a67f506ff93d2 not found: ID does not exist"
Oct 30 22:55:59.186710 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:59.186661    4871 scope.go:122] "RemoveContainer" containerID="3214dd3b7a2df7d71e4ac79864cad8403ab0b9eb2556519fbd6f2d6fe52d17cd"
Oct 30 22:55:59.186990 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:55:59.186968    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3214dd3b7a2df7d71e4ac79864cad8403ab0b9eb2556519fbd6f2d6fe52d17cd\": container with ID starting with 3214dd3b7a2df7d71e4ac79864cad8403ab0b9eb2556519fbd6f2d6fe52d17cd not found: ID does not exist" containerID="3214dd3b7a2df7d71e4ac79864cad8403ab0b9eb2556519fbd6f2d6fe52d17cd"
Oct 30 22:55:59.187047 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:59.187001    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3214dd3b7a2df7d71e4ac79864cad8403ab0b9eb2556519fbd6f2d6fe52d17cd"} err="failed to get container status \"3214dd3b7a2df7d71e4ac79864cad8403ab0b9eb2556519fbd6f2d6fe52d17cd\": rpc error: code = NotFound desc = could not find container \"3214dd3b7a2df7d71e4ac79864cad8403ab0b9eb2556519fbd6f2d6fe52d17cd\": container with ID starting with 3214dd3b7a2df7d71e4ac79864cad8403ab0b9eb2556519fbd6f2d6fe52d17cd not found: ID does not exist"
Oct 30 22:55:59.390860 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:59.390828    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"]
Oct 30 22:55:59.442929 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:59.442840    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-c29159-openshift-default-76db59dbc-dw2q5"]
Oct 30 22:55:59.900443 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:59.900411    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:59.959253 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:55:59.959213    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8acc9cc7-ebae-4d92-8e8c-c8ed8e615030" path="/var/lib/kubelet/pods/8acc9cc7-ebae-4d92-8e8c-c8ed8e615030/volumes"
Oct 30 22:56:00.381709 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:00.381677    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:56:01.932354 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:01.932281    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:56:05.567702 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:56:05.567657    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:56:05.872437 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:05.872399    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-9a5623-597c949cc5-xjcgz"]
Oct 30 22:56:05.982428 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:05.982395    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-9a5623-6fbf554f99-kwmhs"]
Oct 30 22:56:06.040335 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:06.040291    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2205ce10-45gwx"]
Oct 30 22:56:06.040759 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:06.040741    4871 cpu_manager.go:414] "containerMap: removing container" podUID="8acc9cc7-ebae-4d92-8e8c-c8ed8e615030" containerName="istio-proxy"
Oct 30 22:56:06.040854 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:06.040761    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8acc9cc7-ebae-4d92-8e8c-c8ed8e615030" containerName="istio-proxy"
Oct 30 22:56:06.040854 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:06.040802    4871 cpu_manager.go:414] "containerMap: removing container" podUID="a92f92e4-ff49-47c9-9333-beb5ee94a46a" containerName="revoke-keys"
Oct 30 22:56:06.040854 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:06.040811    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a92f92e4-ff49-47c9-9333-beb5ee94a46a" containerName="revoke-keys"
Oct 30 22:56:06.041019 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:06.040941    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="8acc9cc7-ebae-4d92-8e8c-c8ed8e615030" containerName="istio-proxy"
Oct 30 22:56:06.041019 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:06.040959    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="8acc9cc7-ebae-4d92-8e8c-c8ed8e615030" containerName="istio-proxy"
Oct 30 22:56:06.041019 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:06.040970    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="a92f92e4-ff49-47c9-9333-beb5ee94a46a" containerName="revoke-keys"
Oct 30 22:56:06.045851 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:06.045830    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2205ce10-45gwx"
Oct 30 22:56:06.100000 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:06.099968    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2205ce10-45gwx"]
Oct 30 22:56:06.100800 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:06.100778    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:56:06.101004 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:06.100839    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:56:06.101004 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:06.100840    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:56:06.101196 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:06.100915    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mlvtf\""
Oct 30 22:56:06.182918 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:06.182826    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/af4f9127-d296-44c5-98ae-3be64df3df18-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2205ce10-45gwx\" (UID: \"af4f9127-d296-44c5-98ae-3be64df3df18\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2205ce10-45gwx"
Oct 30 22:56:06.183096 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:06.182919    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtr7t\" (UniqueName: \"kubernetes.io/projected/af4f9127-d296-44c5-98ae-3be64df3df18-kube-api-access-qtr7t\") pod \"maas-api-revoke-sub-2205ce10-45gwx\" (UID: \"af4f9127-d296-44c5-98ae-3be64df3df18\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2205ce10-45gwx"
Oct 30 22:56:06.183096 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:06.182951    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/af4f9127-d296-44c5-98ae-3be64df3df18-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2205ce10-45gwx\" (UID: \"af4f9127-d296-44c5-98ae-3be64df3df18\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2205ce10-45gwx"
Oct 30 22:56:06.254576 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:06.254543    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:56:06.284082 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:06.284036    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtr7t\" (UniqueName: \"kubernetes.io/projected/af4f9127-d296-44c5-98ae-3be64df3df18-kube-api-access-qtr7t\") pod \"maas-api-revoke-sub-2205ce10-45gwx\" (UID: \"af4f9127-d296-44c5-98ae-3be64df3df18\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2205ce10-45gwx"
Oct 30 22:56:06.284291 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:06.284091    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/af4f9127-d296-44c5-98ae-3be64df3df18-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2205ce10-45gwx\" (UID: \"af4f9127-d296-44c5-98ae-3be64df3df18\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2205ce10-45gwx"
Oct 30 22:56:06.284291 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:06.284249    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/af4f9127-d296-44c5-98ae-3be64df3df18-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2205ce10-45gwx\" (UID: \"af4f9127-d296-44c5-98ae-3be64df3df18\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2205ce10-45gwx"
Oct 30 22:56:06.286150 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:06.285501    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/af4f9127-d296-44c5-98ae-3be64df3df18-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2205ce10-45gwx\" (UID: \"af4f9127-d296-44c5-98ae-3be64df3df18\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2205ce10-45gwx"
Oct 30 22:56:06.289612 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:06.288910    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/af4f9127-d296-44c5-98ae-3be64df3df18-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2205ce10-45gwx\" (UID: \"af4f9127-d296-44c5-98ae-3be64df3df18\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2205ce10-45gwx"
Oct 30 22:56:06.349048 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:06.349003    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtr7t\" (UniqueName: \"kubernetes.io/projected/af4f9127-d296-44c5-98ae-3be64df3df18-kube-api-access-qtr7t\") pod \"maas-api-revoke-sub-2205ce10-45gwx\" (UID: \"af4f9127-d296-44c5-98ae-3be64df3df18\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2205ce10-45gwx"
Oct 30 22:56:06.359816 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:06.359783    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2205ce10-45gwx"
Oct 30 22:56:06.414741 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:06.406637    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:56:06.738478 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:56:06.738445    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf4f9127_d296_44c5_98ae_3be64df3df18.slice/crio-5f88f8a5679177697740f383ecab498528ada823ace3183fe636b6550c90eac3 WatchSource:0}: Error finding container 5f88f8a5679177697740f383ecab498528ada823ace3183fe636b6550c90eac3: Status 404 returned error can't find the container with id 5f88f8a5679177697740f383ecab498528ada823ace3183fe636b6550c90eac3
Oct 30 22:56:06.742014 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:06.741987    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2205ce10-45gwx"]
Oct 30 22:56:07.218046 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:07.218003    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2205ce10-45gwx" event={"ID":"af4f9127-d296-44c5-98ae-3be64df3df18","Type":"ContainerStarted","Data":"c7ba17f8dee3111cf63c2ea9d38b326cbc45599a041d0998987285eb12c4fa0c"}
Oct 30 22:56:07.218225 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:07.218063    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2205ce10-45gwx" event={"ID":"af4f9127-d296-44c5-98ae-3be64df3df18","Type":"ContainerStarted","Data":"5f88f8a5679177697740f383ecab498528ada823ace3183fe636b6550c90eac3"}
Oct 30 22:56:07.340881 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:07.340772    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2205ce10-45gwx" podStartSLOduration=2.340755965 podStartE2EDuration="2.340755965s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:56:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:56:07.338742876 +0000 UTC m=+4709.516231510" watchObservedRunningTime="2026-10-30 22:56:07.340755965 +0000 UTC m=+4709.518244649"
Oct 30 22:56:08.226448 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:08.226407    4871 generic.go:356] "Generic (PLEG): container finished" podID="af4f9127-d296-44c5-98ae-3be64df3df18" containerID="c7ba17f8dee3111cf63c2ea9d38b326cbc45599a041d0998987285eb12c4fa0c" exitCode=0
Oct 30 22:56:08.226881 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:08.226493    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2205ce10-45gwx" event={"ID":"af4f9127-d296-44c5-98ae-3be64df3df18","Type":"ContainerDied","Data":"c7ba17f8dee3111cf63c2ea9d38b326cbc45599a041d0998987285eb12c4fa0c"}
Oct 30 22:56:09.371822 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:09.371797    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2205ce10-45gwx"
Oct 30 22:56:09.516762 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:09.516652    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/af4f9127-d296-44c5-98ae-3be64df3df18-kube-api-access-qtr7t\" (UniqueName: \"kubernetes.io/projected/af4f9127-d296-44c5-98ae-3be64df3df18-kube-api-access-qtr7t\") pod \"af4f9127-d296-44c5-98ae-3be64df3df18\" (UID: \"af4f9127-d296-44c5-98ae-3be64df3df18\") "
Oct 30 22:56:09.516762 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:09.516702    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/af4f9127-d296-44c5-98ae-3be64df3df18-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/af4f9127-d296-44c5-98ae-3be64df3df18-maas-api-openshift-ca\") pod \"af4f9127-d296-44c5-98ae-3be64df3df18\" (UID: \"af4f9127-d296-44c5-98ae-3be64df3df18\") "
Oct 30 22:56:09.516762 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:09.516740    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/af4f9127-d296-44c5-98ae-3be64df3df18-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/af4f9127-d296-44c5-98ae-3be64df3df18-maas-api-xks-ca\") pod \"af4f9127-d296-44c5-98ae-3be64df3df18\" (UID: \"af4f9127-d296-44c5-98ae-3be64df3df18\") "
Oct 30 22:56:09.517155 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:09.517127    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af4f9127-d296-44c5-98ae-3be64df3df18-maas-api-openshift-ca" pod "af4f9127-d296-44c5-98ae-3be64df3df18" (UID: "af4f9127-d296-44c5-98ae-3be64df3df18"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:56:09.519207 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:09.519160    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af4f9127-d296-44c5-98ae-3be64df3df18-kube-api-access-qtr7t" pod "af4f9127-d296-44c5-98ae-3be64df3df18" (UID: "af4f9127-d296-44c5-98ae-3be64df3df18"). InnerVolumeSpecName "kube-api-access-qtr7t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:56:09.519637 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:09.519600    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af4f9127-d296-44c5-98ae-3be64df3df18-maas-api-xks-ca" pod "af4f9127-d296-44c5-98ae-3be64df3df18" (UID: "af4f9127-d296-44c5-98ae-3be64df3df18"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:56:09.618368 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:09.618321    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qtr7t\" (UniqueName: \"kubernetes.io/projected/af4f9127-d296-44c5-98ae-3be64df3df18-kube-api-access-qtr7t\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:56:09.618368 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:09.618364    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/af4f9127-d296-44c5-98ae-3be64df3df18-maas-api-openshift-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:56:09.618599 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:09.618380    4871 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/af4f9127-d296-44c5-98ae-3be64df3df18-maas-api-xks-ca\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:56:10.243266 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:10.243157    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2205ce10-45gwx"
Oct 30 22:56:10.243266 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:10.243192    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2205ce10-45gwx" event={"ID":"af4f9127-d296-44c5-98ae-3be64df3df18","Type":"ContainerDied","Data":"5f88f8a5679177697740f383ecab498528ada823ace3183fe636b6550c90eac3"}
Oct 30 22:56:10.243548 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:10.243277    4871 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5f88f8a5679177697740f383ecab498528ada823ace3183fe636b6550c90eac3"
Oct 30 22:56:10.873006 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:10.872963    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-9a5623-597c949cc5-xjcgz" podUID="80d68d4f-bef1-438c-a194-45506f62291c" containerName="payload-processing" containerID="cri-o://e3f28b194ce20f54d391eaf76e2c2d15bed5cb344c5021c9cde864f28062c9d2" gracePeriod=25
Oct 30 22:56:10.982779 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:10.982724    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-9a5623-6fbf554f99-kwmhs" podUID="16f9970d-5785-4708-a03a-34fc692d3d2a" containerName="payload-pre-processing" containerID="cri-o://9774fbd0fcd6d672c70abbc260aebd14609eed0846424550948e7252cd180084" gracePeriod=25
Oct 30 22:56:18.248572 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:18.248471    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-9a5623-6fbf554f99-kwmhs" podUID="16f9970d-5785-4708-a03a-34fc692d3d2a" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:56:18.248572 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:18.248512    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-9a5623-597c949cc5-xjcgz" podUID="80d68d4f-bef1-438c-a194-45506f62291c" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:56:20.352337 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:20.352293    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:56:21.929032 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:21.928988    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:56:22.102858 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:22.102816    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:56:22.248032 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:22.247939    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-9a5623-c7cdg"]
Oct 30 22:56:22.300200 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:22.300163    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-9a5623-c7cdg"]
Oct 30 22:56:22.704884 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:22.704846    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:56:23.959852 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:23.959812    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a92f92e4-ff49-47c9-9333-beb5ee94a46a" path="/var/lib/kubelet/pods/a92f92e4-ff49-47c9-9333-beb5ee94a46a/volumes"
Oct 30 22:56:28.248390 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:28.248349    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-9a5623-6fbf554f99-kwmhs" podUID="16f9970d-5785-4708-a03a-34fc692d3d2a" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:56:28.248907 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:28.248350    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-9a5623-597c949cc5-xjcgz" podUID="80d68d4f-bef1-438c-a194-45506f62291c" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:56:28.656894 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:28.656851    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:56:28.831060 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:28.831018    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:56:29.037959 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:29.037885    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"]
Oct 30 22:56:29.038131 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:29.038113    4871 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc" podUID="f506e56e-854d-4a96-a255-f04c7c31a57d" containerName="istio-proxy" containerID="cri-o://b1feeecb95544ceb0f7d8bc42c0d429ecc95e01a57ad527803c500c3d983b47a" gracePeriod=30
Oct 30 22:56:29.070027 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:29.069992    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:56:30.180796 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:30.180761    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-64d8b4548c-bhhfv_89d9923c-dc8d-4655-be5d-ed3c9c36d7b2/authorino/0.log"
Oct 30 22:56:30.309358 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:30.309309    4871 patch_prober.go:29] interesting pod/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:56:30.309563 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:30.309415    4871 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc" podUID="f506e56e-854d-4a96-a255-f04c7c31a57d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:56:30.585658 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:30.585578    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-vqwsv_b87c93be-5f5a-45ba-b820-ebd97c1a1c03/manager/0.log"
Oct 30 22:56:30.829806 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:30.829776    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-p8zgz_9c392070-49cb-4c08-8230-19737bd80de4/manager/0.log"
Oct 30 22:56:31.140827 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:31.140758    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-9a5623-597c949cc5-xjcgz"
Oct 30 22:56:31.233905 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:31.233880    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-9a5623-6fbf554f99-kwmhs"
Oct 30 22:56:31.237865 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:31.237844    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/80d68d4f-bef1-438c-a194-45506f62291c-kube-api-access-4qcjr\" (UniqueName: \"kubernetes.io/projected/80d68d4f-bef1-438c-a194-45506f62291c-kube-api-access-4qcjr\") pod \"80d68d4f-bef1-438c-a194-45506f62291c\" (UID: \"80d68d4f-bef1-438c-a194-45506f62291c\") "
Oct 30 22:56:31.237981 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:31.237962    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/80d68d4f-bef1-438c-a194-45506f62291c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80d68d4f-bef1-438c-a194-45506f62291c-plugins-config-volume\") pod \"80d68d4f-bef1-438c-a194-45506f62291c\" (UID: \"80d68d4f-bef1-438c-a194-45506f62291c\") "
Oct 30 22:56:31.238315 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:31.238292    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80d68d4f-bef1-438c-a194-45506f62291c-plugins-config-volume" pod "80d68d4f-bef1-438c-a194-45506f62291c" (UID: "80d68d4f-bef1-438c-a194-45506f62291c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:56:31.238397 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:31.238380    4871 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80d68d4f-bef1-438c-a194-45506f62291c-plugins-config-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:56:31.240079 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:31.240059    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80d68d4f-bef1-438c-a194-45506f62291c-kube-api-access-4qcjr" pod "80d68d4f-bef1-438c-a194-45506f62291c" (UID: "80d68d4f-bef1-438c-a194-45506f62291c"). InnerVolumeSpecName "kube-api-access-4qcjr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:56:31.339050 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:31.339015    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/16f9970d-5785-4708-a03a-34fc692d3d2a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16f9970d-5785-4708-a03a-34fc692d3d2a-plugins-config-volume\") pod \"16f9970d-5785-4708-a03a-34fc692d3d2a\" (UID: \"16f9970d-5785-4708-a03a-34fc692d3d2a\") "
Oct 30 22:56:31.339258 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:31.339061    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/16f9970d-5785-4708-a03a-34fc692d3d2a-kube-api-access-c4rnc\" (UniqueName: \"kubernetes.io/projected/16f9970d-5785-4708-a03a-34fc692d3d2a-kube-api-access-c4rnc\") pod \"16f9970d-5785-4708-a03a-34fc692d3d2a\" (UID: \"16f9970d-5785-4708-a03a-34fc692d3d2a\") "
Oct 30 22:56:31.339414 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:31.339394    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qcjr\" (UniqueName: \"kubernetes.io/projected/80d68d4f-bef1-438c-a194-45506f62291c-kube-api-access-4qcjr\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:56:31.339458 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:31.339413    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16f9970d-5785-4708-a03a-34fc692d3d2a-plugins-config-volume" pod "16f9970d-5785-4708-a03a-34fc692d3d2a" (UID: "16f9970d-5785-4708-a03a-34fc692d3d2a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:56:31.341197 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:31.341179    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16f9970d-5785-4708-a03a-34fc692d3d2a-kube-api-access-c4rnc" pod "16f9970d-5785-4708-a03a-34fc692d3d2a" (UID: "16f9970d-5785-4708-a03a-34fc692d3d2a"). InnerVolumeSpecName "kube-api-access-c4rnc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:56:31.373898 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:31.373867    4871 generic.go:356] "Generic (PLEG): container finished" podID="80d68d4f-bef1-438c-a194-45506f62291c" containerID="e3f28b194ce20f54d391eaf76e2c2d15bed5cb344c5021c9cde864f28062c9d2" exitCode=0
Oct 30 22:56:31.374082 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:31.373915    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-9a5623-597c949cc5-xjcgz"
Oct 30 22:56:31.374082 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:31.373950    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-9a5623-597c949cc5-xjcgz" event={"ID":"80d68d4f-bef1-438c-a194-45506f62291c","Type":"ContainerDied","Data":"e3f28b194ce20f54d391eaf76e2c2d15bed5cb344c5021c9cde864f28062c9d2"}
Oct 30 22:56:31.374082 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:31.373999    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-9a5623-597c949cc5-xjcgz" event={"ID":"80d68d4f-bef1-438c-a194-45506f62291c","Type":"ContainerDied","Data":"d9ea43ed7ea81d0d9abbf41d977a1d8f5ab3516bc4864e9a6cc745d82dbe861d"}
Oct 30 22:56:31.374082 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:31.374040    4871 scope.go:122] "RemoveContainer" containerID="e3f28b194ce20f54d391eaf76e2c2d15bed5cb344c5021c9cde864f28062c9d2"
Oct 30 22:56:31.375609 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:31.375587    4871 generic.go:356] "Generic (PLEG): container finished" podID="16f9970d-5785-4708-a03a-34fc692d3d2a" containerID="9774fbd0fcd6d672c70abbc260aebd14609eed0846424550948e7252cd180084" exitCode=0
Oct 30 22:56:31.375739 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:31.375642    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-9a5623-6fbf554f99-kwmhs" event={"ID":"16f9970d-5785-4708-a03a-34fc692d3d2a","Type":"ContainerDied","Data":"9774fbd0fcd6d672c70abbc260aebd14609eed0846424550948e7252cd180084"}
Oct 30 22:56:31.375739 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:31.375677    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-9a5623-6fbf554f99-kwmhs" event={"ID":"16f9970d-5785-4708-a03a-34fc692d3d2a","Type":"ContainerDied","Data":"3da8f74e77d7f6e6981e7e98861e294303944a3554052789a19cd1da0c691df0"}
Oct 30 22:56:31.375739 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:31.375643    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-9a5623-6fbf554f99-kwmhs"
Oct 30 22:56:31.385003 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:31.384905    4871 scope.go:122] "RemoveContainer" containerID="e3f28b194ce20f54d391eaf76e2c2d15bed5cb344c5021c9cde864f28062c9d2"
Oct 30 22:56:31.388691 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:56:31.388538    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3f28b194ce20f54d391eaf76e2c2d15bed5cb344c5021c9cde864f28062c9d2\": container with ID starting with e3f28b194ce20f54d391eaf76e2c2d15bed5cb344c5021c9cde864f28062c9d2 not found: ID does not exist" containerID="e3f28b194ce20f54d391eaf76e2c2d15bed5cb344c5021c9cde864f28062c9d2"
Oct 30 22:56:31.388691 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:31.388581    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3f28b194ce20f54d391eaf76e2c2d15bed5cb344c5021c9cde864f28062c9d2"} err="failed to get container status \"e3f28b194ce20f54d391eaf76e2c2d15bed5cb344c5021c9cde864f28062c9d2\": rpc error: code = NotFound desc = could not find container \"e3f28b194ce20f54d391eaf76e2c2d15bed5cb344c5021c9cde864f28062c9d2\": container with ID starting with e3f28b194ce20f54d391eaf76e2c2d15bed5cb344c5021c9cde864f28062c9d2 not found: ID does not exist"
Oct 30 22:56:31.388691 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:31.388606    4871 scope.go:122] "RemoveContainer" containerID="9774fbd0fcd6d672c70abbc260aebd14609eed0846424550948e7252cd180084"
Oct 30 22:56:31.406921 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:31.400741    4871 scope.go:122] "RemoveContainer" containerID="9774fbd0fcd6d672c70abbc260aebd14609eed0846424550948e7252cd180084"
Oct 30 22:56:31.407486 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:56:31.407405    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9774fbd0fcd6d672c70abbc260aebd14609eed0846424550948e7252cd180084\": container with ID starting with 9774fbd0fcd6d672c70abbc260aebd14609eed0846424550948e7252cd180084 not found: ID does not exist" containerID="9774fbd0fcd6d672c70abbc260aebd14609eed0846424550948e7252cd180084"
Oct 30 22:56:31.407486 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:31.407448    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9774fbd0fcd6d672c70abbc260aebd14609eed0846424550948e7252cd180084"} err="failed to get container status \"9774fbd0fcd6d672c70abbc260aebd14609eed0846424550948e7252cd180084\": rpc error: code = NotFound desc = could not find container \"9774fbd0fcd6d672c70abbc260aebd14609eed0846424550948e7252cd180084\": container with ID starting with 9774fbd0fcd6d672c70abbc260aebd14609eed0846424550948e7252cd180084 not found: ID does not exist"
Oct 30 22:56:31.440650 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:31.440618    4871 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16f9970d-5785-4708-a03a-34fc692d3d2a-plugins-config-volume\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:56:31.440650 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:31.440655    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c4rnc\" (UniqueName: \"kubernetes.io/projected/16f9970d-5785-4708-a03a-34fc692d3d2a-kube-api-access-c4rnc\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:56:31.526849 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:31.526806    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-9a5623-6fbf554f99-kwmhs"]
Oct 30 22:56:31.531315 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:31.531284    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-9a5623-6fbf554f99-kwmhs"]
Oct 30 22:56:31.648262 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:31.648209    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-9a5623-597c949cc5-xjcgz"]
Oct 30 22:56:31.650928 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:31.650878    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-9a5623-597c949cc5-xjcgz"]
Oct 30 22:56:31.961203 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:31.961120    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="16f9970d-5785-4708-a03a-34fc692d3d2a" path="/var/lib/kubelet/pods/16f9970d-5785-4708-a03a-34fc692d3d2a/volumes"
Oct 30 22:56:31.961665 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:31.961642    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="80d68d4f-bef1-438c-a194-45506f62291c" path="/var/lib/kubelet/pods/80d68d4f-bef1-438c-a194-45506f62291c/volumes"
Oct 30 22:56:32.301721 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:32.301625    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-r5c8j_4f069634-b231-4fd0-847e-52cb47ef0163/manager/0.log"
Oct 30 22:56:32.776665 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:32.776610    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-pfgpx_5754b4e2-2ccc-43ee-b1d2-e065d15345ca/manager/0.log"
Oct 30 22:56:34.292868 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:34.292840    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"
Oct 30 22:56:34.369671 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:34.369639    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f506e56e-854d-4a96-a255-f04c7c31a57d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f506e56e-854d-4a96-a255-f04c7c31a57d-istio-envoy\") pod \"f506e56e-854d-4a96-a255-f04c7c31a57d\" (UID: \"f506e56e-854d-4a96-a255-f04c7c31a57d\") "
Oct 30 22:56:34.369872 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:34.369689    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f506e56e-854d-4a96-a255-f04c7c31a57d-istio-token\" (UniqueName: \"kubernetes.io/projected/f506e56e-854d-4a96-a255-f04c7c31a57d-istio-token\") pod \"f506e56e-854d-4a96-a255-f04c7c31a57d\" (UID: \"f506e56e-854d-4a96-a255-f04c7c31a57d\") "
Oct 30 22:56:34.369872 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:34.369713    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f506e56e-854d-4a96-a255-f04c7c31a57d-kube-api-access-5whcd\" (UniqueName: \"kubernetes.io/projected/f506e56e-854d-4a96-a255-f04c7c31a57d-kube-api-access-5whcd\") pod \"f506e56e-854d-4a96-a255-f04c7c31a57d\" (UID: \"f506e56e-854d-4a96-a255-f04c7c31a57d\") "
Oct 30 22:56:34.369872 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:34.369732    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f506e56e-854d-4a96-a255-f04c7c31a57d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f506e56e-854d-4a96-a255-f04c7c31a57d-istiod-ca-cert\") pod \"f506e56e-854d-4a96-a255-f04c7c31a57d\" (UID: \"f506e56e-854d-4a96-a255-f04c7c31a57d\") "
Oct 30 22:56:34.369872 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:34.369757    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f506e56e-854d-4a96-a255-f04c7c31a57d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f506e56e-854d-4a96-a255-f04c7c31a57d-workload-certs\") pod \"f506e56e-854d-4a96-a255-f04c7c31a57d\" (UID: \"f506e56e-854d-4a96-a255-f04c7c31a57d\") "
Oct 30 22:56:34.369872 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:34.369797    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f506e56e-854d-4a96-a255-f04c7c31a57d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f506e56e-854d-4a96-a255-f04c7c31a57d-istio-podinfo\") pod \"f506e56e-854d-4a96-a255-f04c7c31a57d\" (UID: \"f506e56e-854d-4a96-a255-f04c7c31a57d\") "
Oct 30 22:56:34.369872 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:34.369853    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f506e56e-854d-4a96-a255-f04c7c31a57d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f506e56e-854d-4a96-a255-f04c7c31a57d-istio-data\") pod \"f506e56e-854d-4a96-a255-f04c7c31a57d\" (UID: \"f506e56e-854d-4a96-a255-f04c7c31a57d\") "
Oct 30 22:56:34.370228 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:34.369890    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f506e56e-854d-4a96-a255-f04c7c31a57d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f506e56e-854d-4a96-a255-f04c7c31a57d-credential-socket\") pod \"f506e56e-854d-4a96-a255-f04c7c31a57d\" (UID: \"f506e56e-854d-4a96-a255-f04c7c31a57d\") "
Oct 30 22:56:34.370228 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:34.369920    4871 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f506e56e-854d-4a96-a255-f04c7c31a57d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f506e56e-854d-4a96-a255-f04c7c31a57d-workload-socket\") pod \"f506e56e-854d-4a96-a255-f04c7c31a57d\" (UID: \"f506e56e-854d-4a96-a255-f04c7c31a57d\") "
Oct 30 22:56:34.370228 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:34.370129    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f506e56e-854d-4a96-a255-f04c7c31a57d-workload-certs" pod "f506e56e-854d-4a96-a255-f04c7c31a57d" (UID: "f506e56e-854d-4a96-a255-f04c7c31a57d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:56:34.370228 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:34.370217    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f506e56e-854d-4a96-a255-f04c7c31a57d-istiod-ca-cert" pod "f506e56e-854d-4a96-a255-f04c7c31a57d" (UID: "f506e56e-854d-4a96-a255-f04c7c31a57d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:56:34.370501 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:34.370391    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f506e56e-854d-4a96-a255-f04c7c31a57d-workload-socket" pod "f506e56e-854d-4a96-a255-f04c7c31a57d" (UID: "f506e56e-854d-4a96-a255-f04c7c31a57d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:56:34.370501 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:34.370436    4871 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f506e56e-854d-4a96-a255-f04c7c31a57d-istiod-ca-cert\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:56:34.370501 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:34.370458    4871 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f506e56e-854d-4a96-a255-f04c7c31a57d-workload-certs\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:56:34.370501 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:34.370455    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f506e56e-854d-4a96-a255-f04c7c31a57d-credential-socket" pod "f506e56e-854d-4a96-a255-f04c7c31a57d" (UID: "f506e56e-854d-4a96-a255-f04c7c31a57d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:56:34.370739 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:34.370500    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f506e56e-854d-4a96-a255-f04c7c31a57d-istio-data" pod "f506e56e-854d-4a96-a255-f04c7c31a57d" (UID: "f506e56e-854d-4a96-a255-f04c7c31a57d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:56:34.371982 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:34.371953    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f506e56e-854d-4a96-a255-f04c7c31a57d-istio-envoy" pod "f506e56e-854d-4a96-a255-f04c7c31a57d" (UID: "f506e56e-854d-4a96-a255-f04c7c31a57d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:56:34.372098 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:34.372006    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f506e56e-854d-4a96-a255-f04c7c31a57d-istio-token" pod "f506e56e-854d-4a96-a255-f04c7c31a57d" (UID: "f506e56e-854d-4a96-a255-f04c7c31a57d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:56:34.372098 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:34.372081    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f506e56e-854d-4a96-a255-f04c7c31a57d-istio-podinfo" pod "f506e56e-854d-4a96-a255-f04c7c31a57d" (UID: "f506e56e-854d-4a96-a255-f04c7c31a57d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:56:34.372098 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:34.372086    4871 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f506e56e-854d-4a96-a255-f04c7c31a57d-kube-api-access-5whcd" pod "f506e56e-854d-4a96-a255-f04c7c31a57d" (UID: "f506e56e-854d-4a96-a255-f04c7c31a57d"). InnerVolumeSpecName "kube-api-access-5whcd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:56:34.400032 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:34.399998    4871 generic.go:356] "Generic (PLEG): container finished" podID="f506e56e-854d-4a96-a255-f04c7c31a57d" containerID="b1feeecb95544ceb0f7d8bc42c0d429ecc95e01a57ad527803c500c3d983b47a" exitCode=0
Oct 30 22:56:34.400199 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:34.400051    4871 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"
Oct 30 22:56:34.400199 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:34.400117    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc" event={"ID":"f506e56e-854d-4a96-a255-f04c7c31a57d","Type":"ContainerDied","Data":"b1feeecb95544ceb0f7d8bc42c0d429ecc95e01a57ad527803c500c3d983b47a"}
Oct 30 22:56:34.400199 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:34.400181    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc" event={"ID":"f506e56e-854d-4a96-a255-f04c7c31a57d","Type":"ContainerDied","Data":"66602091e4511816fafc75a5fb9e9923c80c076440f01b97cc7fa0047afe885a"}
Oct 30 22:56:34.400367 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:34.400210    4871 scope.go:122] "RemoveContainer" containerID="b1feeecb95544ceb0f7d8bc42c0d429ecc95e01a57ad527803c500c3d983b47a"
Oct 30 22:56:34.410422 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:34.410396    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-qzpg5_b7de093c-a605-4406-a940-e3e59abcb9a0/manager/0.log"
Oct 30 22:56:34.411928 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:34.411909    4871 scope.go:122] "RemoveContainer" containerID="b1feeecb95544ceb0f7d8bc42c0d429ecc95e01a57ad527803c500c3d983b47a"
Oct 30 22:56:34.412218 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:56:34.412195    4871 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1feeecb95544ceb0f7d8bc42c0d429ecc95e01a57ad527803c500c3d983b47a\": container with ID starting with b1feeecb95544ceb0f7d8bc42c0d429ecc95e01a57ad527803c500c3d983b47a not found: ID does not exist" containerID="b1feeecb95544ceb0f7d8bc42c0d429ecc95e01a57ad527803c500c3d983b47a"
Oct 30 22:56:34.412399 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:34.412230    4871 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1feeecb95544ceb0f7d8bc42c0d429ecc95e01a57ad527803c500c3d983b47a"} err="failed to get container status \"b1feeecb95544ceb0f7d8bc42c0d429ecc95e01a57ad527803c500c3d983b47a\": rpc error: code = NotFound desc = could not find container \"b1feeecb95544ceb0f7d8bc42c0d429ecc95e01a57ad527803c500c3d983b47a\": container with ID starting with b1feeecb95544ceb0f7d8bc42c0d429ecc95e01a57ad527803c500c3d983b47a not found: ID does not exist"
Oct 30 22:56:34.470982 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:34.470946    4871 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f506e56e-854d-4a96-a255-f04c7c31a57d-istio-envoy\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:56:34.470982 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:34.470976    4871 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f506e56e-854d-4a96-a255-f04c7c31a57d-istio-token\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:56:34.471205 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:34.470992    4871 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5whcd\" (UniqueName: \"kubernetes.io/projected/f506e56e-854d-4a96-a255-f04c7c31a57d-kube-api-access-5whcd\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:56:34.471205 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:34.471008    4871 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f506e56e-854d-4a96-a255-f04c7c31a57d-istio-podinfo\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:56:34.471205 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:34.471020    4871 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f506e56e-854d-4a96-a255-f04c7c31a57d-istio-data\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:56:34.471205 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:34.471032    4871 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f506e56e-854d-4a96-a255-f04c7c31a57d-credential-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:56:34.471205 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:34.471043    4871 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f506e56e-854d-4a96-a255-f04c7c31a57d-workload-socket\") on node \"ip-10-0-139-46.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:56:34.588945 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:34.588899    4871 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"]
Oct 30 22:56:34.595544 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:34.595514    4871 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-9a5623-openshift-default-5455ff6b5c-t5zxc"]
Oct 30 22:56:35.093901 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:35.093856    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:56:35.120926 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:35.120897    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-84769bb5bf-j5wbn_888e372b-f344-412f-9f59-6411a4f1177e/manager/0.log"
Oct 30 22:56:35.267666 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:35.267581    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:56:35.568199 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:56:35.568162    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:56:35.958683 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:35.958649    4871 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f506e56e-854d-4a96-a255-f04c7c31a57d" path="/var/lib/kubelet/pods/f506e56e-854d-4a96-a255-f04c7c31a57d/volumes"
Oct 30 22:56:36.648620 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:36.648585    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-64d8b4548c-bhhfv_89d9923c-dc8d-4655-be5d-ed3c9c36d7b2/authorino/0.log"
Oct 30 22:56:36.827766 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:36.827717    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:56:38.093211 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:38.093170    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:56:38.293816 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:38.293763    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:56:42.870148 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:42.870113    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:56:48.061578 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:48.061545    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-qzpg5_b7de093c-a605-4406-a940-e3e59abcb9a0/manager/0.log"
Oct 30 22:56:48.854628 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:48.854591    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-84769bb5bf-j5wbn_888e372b-f344-412f-9f59-6411a4f1177e/manager/0.log"
Oct 30 22:56:51.068971 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:51.068933    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-2205ce10-45gwx_af4f9127-d296-44c5-98ae-3be64df3df18/revoke-keys/0.log"
Oct 30 22:56:53.998036 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:53.998003    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-64d8b4548c-bhhfv_89d9923c-dc8d-4655-be5d-ed3c9c36d7b2/authorino/0.log"
Oct 30 22:56:54.408911 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:54.408876    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-vqwsv_b87c93be-5f5a-45ba-b820-ebd97c1a1c03/manager/0.log"
Oct 30 22:56:54.647533 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:54.647500    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-p8zgz_9c392070-49cb-4c08-8230-19737bd80de4/manager/0.log"
Oct 30 22:56:55.933918 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:55.933876    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-r5c8j_4f069634-b231-4fd0-847e-52cb47ef0163/manager/0.log"
Oct 30 22:56:56.220789 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:56.220699    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:56:56.400925 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:56.400891    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-pfgpx_5754b4e2-2ccc-43ee-b1d2-e065d15345ca/manager/0.log"
Oct 30 22:56:56.403278 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:56.403224    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:56:57.247587 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:57.247557    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-575956cc9c6mmf7_3419e5d7-f2ed-43e1-89e0-baccdb485f1b/istio-proxy/0.log"
Oct 30 22:56:57.484940 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:57.484907    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-l8t6z_1817be16-1033-45eb-8499-5c2d201bbc2e/discovery/0.log"
Oct 30 22:56:57.659858 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:57.659818    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:56:58.200146 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:58.200112    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7df854bf67-vjrqn_cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d/istio-proxy/0.log"
Oct 30 22:56:58.425360 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:58.425312    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-668d64f647-59zws_54b7cb62-6e43-40d4-afbb-2267c98b8c52/istio-proxy/0.log"
Oct 30 22:56:59.126084 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:56:59.126048    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-57f669cfc9-d755h_e0535562-96d7-4fc4-a5d5-ad4cc785080e/router/0.log"
Oct 30 22:57:01.448629 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:01.448594    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-79krz_e0a4171a-a8b7-45be-8bcc-8bc8d598e091/main/0.log"
Oct 30 22:57:05.568468 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:57:05.568432    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:57:12.664555 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:12.664517    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:57:12.733753 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:12.733716    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-v9zbs/must-gather-gg7m2"]
Oct 30 22:57:12.734128 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:12.734114    4871 cpu_manager.go:414] "containerMap: removing container" podUID="af4f9127-d296-44c5-98ae-3be64df3df18" containerName="revoke-keys"
Oct 30 22:57:12.734174 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:12.734130    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="af4f9127-d296-44c5-98ae-3be64df3df18" containerName="revoke-keys"
Oct 30 22:57:12.734174 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:12.734143    4871 cpu_manager.go:414] "containerMap: removing container" podUID="8acc9cc7-ebae-4d92-8e8c-c8ed8e615030" containerName="istio-proxy"
Oct 30 22:57:12.734174 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:12.734151    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8acc9cc7-ebae-4d92-8e8c-c8ed8e615030" containerName="istio-proxy"
Oct 30 22:57:12.734174 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:12.734170    4871 cpu_manager.go:414] "containerMap: removing container" podUID="16f9970d-5785-4708-a03a-34fc692d3d2a" containerName="payload-pre-processing"
Oct 30 22:57:12.734359 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:12.734175    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="16f9970d-5785-4708-a03a-34fc692d3d2a" containerName="payload-pre-processing"
Oct 30 22:57:12.734359 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:12.734185    4871 cpu_manager.go:414] "containerMap: removing container" podUID="80d68d4f-bef1-438c-a194-45506f62291c" containerName="payload-processing"
Oct 30 22:57:12.734359 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:12.734190    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="80d68d4f-bef1-438c-a194-45506f62291c" containerName="payload-processing"
Oct 30 22:57:12.734359 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:12.734203    4871 cpu_manager.go:414] "containerMap: removing container" podUID="f506e56e-854d-4a96-a255-f04c7c31a57d" containerName="istio-proxy"
Oct 30 22:57:12.734359 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:12.734207    4871 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f506e56e-854d-4a96-a255-f04c7c31a57d" containerName="istio-proxy"
Oct 30 22:57:12.734359 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:12.734298    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="af4f9127-d296-44c5-98ae-3be64df3df18" containerName="revoke-keys"
Oct 30 22:57:12.734359 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:12.734313    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="16f9970d-5785-4708-a03a-34fc692d3d2a" containerName="payload-pre-processing"
Oct 30 22:57:12.734359 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:12.734321    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="80d68d4f-bef1-438c-a194-45506f62291c" containerName="payload-processing"
Oct 30 22:57:12.734359 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:12.734328    4871 memory_manager.go:365] "RemoveStaleState removing state" podUID="f506e56e-854d-4a96-a255-f04c7c31a57d" containerName="istio-proxy"
Oct 30 22:57:12.737538 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:12.737520    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-v9zbs/must-gather-gg7m2"
Oct 30 22:57:12.798408 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:12.798366    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-v9zbs/must-gather-gg7m2"]
Oct 30 22:57:12.834710 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:12.834672    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:57:12.844721 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:12.844688    4871 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-v9zbs\"/\"default-dockercfg-r6fhw\""
Oct 30 22:57:12.844721 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:12.844716    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-v9zbs\"/\"kube-root-ca.crt\""
Oct 30 22:57:12.844915 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:12.844694    4871 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-v9zbs\"/\"openshift-service-ca.crt\""
Oct 30 22:57:12.906400 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:12.906358    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c8ee3963-81d4-4c3e-9a87-f734456f038f-must-gather-output\") pod \"must-gather-gg7m2\" (UID: \"c8ee3963-81d4-4c3e-9a87-f734456f038f\") " pod="openshift-must-gather-v9zbs/must-gather-gg7m2"
Oct 30 22:57:12.906593 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:12.906484    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5dcp\" (UniqueName: \"kubernetes.io/projected/c8ee3963-81d4-4c3e-9a87-f734456f038f-kube-api-access-r5dcp\") pod \"must-gather-gg7m2\" (UID: \"c8ee3963-81d4-4c3e-9a87-f734456f038f\") " pod="openshift-must-gather-v9zbs/must-gather-gg7m2"
Oct 30 22:57:13.006988 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:13.006890    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5dcp\" (UniqueName: \"kubernetes.io/projected/c8ee3963-81d4-4c3e-9a87-f734456f038f-kube-api-access-r5dcp\") pod \"must-gather-gg7m2\" (UID: \"c8ee3963-81d4-4c3e-9a87-f734456f038f\") " pod="openshift-must-gather-v9zbs/must-gather-gg7m2"
Oct 30 22:57:13.006988 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:13.006974    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c8ee3963-81d4-4c3e-9a87-f734456f038f-must-gather-output\") pod \"must-gather-gg7m2\" (UID: \"c8ee3963-81d4-4c3e-9a87-f734456f038f\") " pod="openshift-must-gather-v9zbs/must-gather-gg7m2"
Oct 30 22:57:13.007343 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:13.007323    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c8ee3963-81d4-4c3e-9a87-f734456f038f-must-gather-output\") pod \"must-gather-gg7m2\" (UID: \"c8ee3963-81d4-4c3e-9a87-f734456f038f\") " pod="openshift-must-gather-v9zbs/must-gather-gg7m2"
Oct 30 22:57:13.072597 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:13.072566    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5dcp\" (UniqueName: \"kubernetes.io/projected/c8ee3963-81d4-4c3e-9a87-f734456f038f-kube-api-access-r5dcp\") pod \"must-gather-gg7m2\" (UID: \"c8ee3963-81d4-4c3e-9a87-f734456f038f\") " pod="openshift-must-gather-v9zbs/must-gather-gg7m2"
Oct 30 22:57:13.351154 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:13.351065    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-v9zbs/must-gather-gg7m2"
Oct 30 22:57:13.723361 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:57:13.723321    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc8ee3963_81d4_4c3e_9a87_f734456f038f.slice/crio-27c64c8b7923e9182e13c03cdfad8c7d559921e48fee242e7b4bb4c18a2e2289 WatchSource:0}: Error finding container 27c64c8b7923e9182e13c03cdfad8c7d559921e48fee242e7b4bb4c18a2e2289: Status 404 returned error can't find the container with id 27c64c8b7923e9182e13c03cdfad8c7d559921e48fee242e7b4bb4c18a2e2289
Oct 30 22:57:13.725435 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:13.725417    4871 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:57:13.727109 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:13.727083    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-v9zbs/must-gather-gg7m2"]
Oct 30 22:57:14.634856 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:14.634813    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v9zbs/must-gather-gg7m2" event={"ID":"c8ee3963-81d4-4c3e-9a87-f734456f038f","Type":"ContainerStarted","Data":"27c64c8b7923e9182e13c03cdfad8c7d559921e48fee242e7b4bb4c18a2e2289"}
Oct 30 22:57:15.643580 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:15.643543    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v9zbs/must-gather-gg7m2" event={"ID":"c8ee3963-81d4-4c3e-9a87-f734456f038f","Type":"ContainerStarted","Data":"0e44413b2a891fe7faf3e0a75a5d44c6c5027705e34cbb0f08df04fe245fd849"}
Oct 30 22:57:15.643580 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:15.643587    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v9zbs/must-gather-gg7m2" event={"ID":"c8ee3963-81d4-4c3e-9a87-f734456f038f","Type":"ContainerStarted","Data":"b0057c6c7b29bd09524d1e77d27f7d7bff3066ba9e7793ae498e514eb2cc4040"}
Oct 30 22:57:15.765458 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:15.765398    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-v9zbs/must-gather-gg7m2" podStartSLOduration=2.373896767 podStartE2EDuration="3.765382856s" totalImagesPullingTime="1.391486089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:57:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:57:15.764333897 +0000 UTC m=+4777.941822515" watchObservedRunningTime="2026-10-30 22:57:15.765382856 +0000 UTC m=+4777.942871473"
Oct 30 22:57:19.153453 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:19.153413    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:57:19.348727 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:19.348692    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:57:22.894478 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:22.894445    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-z9h2r_6e615be3-41d7-4f82-90d7-7ce71ed9190b/global-pull-secret-syncer/0.log"
Oct 30 22:57:23.511260 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:23.510674    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-nvwdl_d54af961-c593-4707-9cec-1069000bce50/connection-checker/0.log"
Oct 30 22:57:23.929720 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:23.929562    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-px9dx_5f5757a0-0a11-4637-9aa5-4e283792017d/connection-checker/0.log"
Oct 30 22:57:24.352119 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:24.352032    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-tlfjh_1dac48d8-aefb-4ff4-9213-b13def641c00/connection-checker/0.log"
Oct 30 22:57:24.810768 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:24.810733    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log"
Oct 30 22:57:26.432258 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:26.428833    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-qzpg5_b7de093c-a605-4406-a940-e3e59abcb9a0/manager/0.log"
Oct 30 22:57:26.579671 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:26.579633    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-46.us-west-2.compute.internal_1d836aa5280c51c365d2d598465f9d5c/haproxy/0.log"
Oct 30 22:57:26.610893 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:26.610854    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-qzpg5_b7de093c-a605-4406-a940-e3e59abcb9a0/manager/0.log"
Oct 30 22:57:27.594525 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:27.594445    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-84769bb5bf-j5wbn_888e372b-f344-412f-9f59-6411a4f1177e/manager/0.log"
Oct 30 22:57:27.937341 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:27.937302    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-84769bb5bf-j5wbn_888e372b-f344-412f-9f59-6411a4f1177e/manager/0.log"
Oct 30 22:57:32.071010 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:32.070971    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-64d8b4548c-bhhfv_89d9923c-dc8d-4655-be5d-ed3c9c36d7b2/authorino/0.log"
Oct 30 22:57:32.390655 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:32.390618    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-vqwsv_b87c93be-5f5a-45ba-b820-ebd97c1a1c03/manager/0.log"
Oct 30 22:57:32.759619 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:32.759530    4871 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-v9zbs/perf-node-gather-daemonset-4l9ph"]
Oct 30 22:57:32.765490 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:32.765451    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-v9zbs/perf-node-gather-daemonset-4l9ph"
Oct 30 22:57:32.806748 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:32.805704    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/551da8e0-4504-4c15-910c-d88c9aa223fc-lib-modules\") pod \"perf-node-gather-daemonset-4l9ph\" (UID: \"551da8e0-4504-4c15-910c-d88c9aa223fc\") " pod="openshift-must-gather-v9zbs/perf-node-gather-daemonset-4l9ph"
Oct 30 22:57:32.806748 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:32.805776    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/551da8e0-4504-4c15-910c-d88c9aa223fc-sys\") pod \"perf-node-gather-daemonset-4l9ph\" (UID: \"551da8e0-4504-4c15-910c-d88c9aa223fc\") " pod="openshift-must-gather-v9zbs/perf-node-gather-daemonset-4l9ph"
Oct 30 22:57:32.806748 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:32.805835    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/551da8e0-4504-4c15-910c-d88c9aa223fc-proc\") pod \"perf-node-gather-daemonset-4l9ph\" (UID: \"551da8e0-4504-4c15-910c-d88c9aa223fc\") " pod="openshift-must-gather-v9zbs/perf-node-gather-daemonset-4l9ph"
Oct 30 22:57:32.806748 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:32.805925    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdklk\" (UniqueName: \"kubernetes.io/projected/551da8e0-4504-4c15-910c-d88c9aa223fc-kube-api-access-mdklk\") pod \"perf-node-gather-daemonset-4l9ph\" (UID: \"551da8e0-4504-4c15-910c-d88c9aa223fc\") " pod="openshift-must-gather-v9zbs/perf-node-gather-daemonset-4l9ph"
Oct 30 22:57:32.806748 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:32.805969    4871 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/551da8e0-4504-4c15-910c-d88c9aa223fc-podres\") pod \"perf-node-gather-daemonset-4l9ph\" (UID: \"551da8e0-4504-4c15-910c-d88c9aa223fc\") " pod="openshift-must-gather-v9zbs/perf-node-gather-daemonset-4l9ph"
Oct 30 22:57:32.839907 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:32.839880    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-p8zgz_9c392070-49cb-4c08-8230-19737bd80de4/manager/0.log"
Oct 30 22:57:32.845039 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:32.845009    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-v9zbs/perf-node-gather-daemonset-4l9ph"]
Oct 30 22:57:32.906483 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:32.906444    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdklk\" (UniqueName: \"kubernetes.io/projected/551da8e0-4504-4c15-910c-d88c9aa223fc-kube-api-access-mdklk\") pod \"perf-node-gather-daemonset-4l9ph\" (UID: \"551da8e0-4504-4c15-910c-d88c9aa223fc\") " pod="openshift-must-gather-v9zbs/perf-node-gather-daemonset-4l9ph"
Oct 30 22:57:32.906483 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:32.906489    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/551da8e0-4504-4c15-910c-d88c9aa223fc-podres\") pod \"perf-node-gather-daemonset-4l9ph\" (UID: \"551da8e0-4504-4c15-910c-d88c9aa223fc\") " pod="openshift-must-gather-v9zbs/perf-node-gather-daemonset-4l9ph"
Oct 30 22:57:32.906724 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:32.906515    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/551da8e0-4504-4c15-910c-d88c9aa223fc-lib-modules\") pod \"perf-node-gather-daemonset-4l9ph\" (UID: \"551da8e0-4504-4c15-910c-d88c9aa223fc\") " pod="openshift-must-gather-v9zbs/perf-node-gather-daemonset-4l9ph"
Oct 30 22:57:32.906724 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:32.906552    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/551da8e0-4504-4c15-910c-d88c9aa223fc-sys\") pod \"perf-node-gather-daemonset-4l9ph\" (UID: \"551da8e0-4504-4c15-910c-d88c9aa223fc\") " pod="openshift-must-gather-v9zbs/perf-node-gather-daemonset-4l9ph"
Oct 30 22:57:32.906724 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:32.906588    4871 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/551da8e0-4504-4c15-910c-d88c9aa223fc-proc\") pod \"perf-node-gather-daemonset-4l9ph\" (UID: \"551da8e0-4504-4c15-910c-d88c9aa223fc\") " pod="openshift-must-gather-v9zbs/perf-node-gather-daemonset-4l9ph"
Oct 30 22:57:32.906724 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:32.906599    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/551da8e0-4504-4c15-910c-d88c9aa223fc-podres\") pod \"perf-node-gather-daemonset-4l9ph\" (UID: \"551da8e0-4504-4c15-910c-d88c9aa223fc\") " pod="openshift-must-gather-v9zbs/perf-node-gather-daemonset-4l9ph"
Oct 30 22:57:32.906724 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:32.906671    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/551da8e0-4504-4c15-910c-d88c9aa223fc-proc\") pod \"perf-node-gather-daemonset-4l9ph\" (UID: \"551da8e0-4504-4c15-910c-d88c9aa223fc\") " pod="openshift-must-gather-v9zbs/perf-node-gather-daemonset-4l9ph"
Oct 30 22:57:32.906724 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:32.906671    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/551da8e0-4504-4c15-910c-d88c9aa223fc-lib-modules\") pod \"perf-node-gather-daemonset-4l9ph\" (UID: \"551da8e0-4504-4c15-910c-d88c9aa223fc\") " pod="openshift-must-gather-v9zbs/perf-node-gather-daemonset-4l9ph"
Oct 30 22:57:32.906724 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:32.906673    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/551da8e0-4504-4c15-910c-d88c9aa223fc-sys\") pod \"perf-node-gather-daemonset-4l9ph\" (UID: \"551da8e0-4504-4c15-910c-d88c9aa223fc\") " pod="openshift-must-gather-v9zbs/perf-node-gather-daemonset-4l9ph"
Oct 30 22:57:32.970610 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:32.970563    4871 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdklk\" (UniqueName: \"kubernetes.io/projected/551da8e0-4504-4c15-910c-d88c9aa223fc-kube-api-access-mdklk\") pod \"perf-node-gather-daemonset-4l9ph\" (UID: \"551da8e0-4504-4c15-910c-d88c9aa223fc\") " pod="openshift-must-gather-v9zbs/perf-node-gather-daemonset-4l9ph"
Oct 30 22:57:33.084433 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:33.084349    4871 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-v9zbs/perf-node-gather-daemonset-4l9ph"
Oct 30 22:57:33.459538 ip-10-0-139-46 kubenswrapper[4871]: W1030 22:57:33.459491    4871 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod551da8e0_4504_4c15_910c_d88c9aa223fc.slice/crio-e22d10b7e16cffeef032f1c2019e86d49ff1184a30fa41cb539f0030aa1f599d WatchSource:0}: Error finding container e22d10b7e16cffeef032f1c2019e86d49ff1184a30fa41cb539f0030aa1f599d: Status 404 returned error can't find the container with id e22d10b7e16cffeef032f1c2019e86d49ff1184a30fa41cb539f0030aa1f599d
Oct 30 22:57:33.463106 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:33.463081    4871 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-v9zbs/perf-node-gather-daemonset-4l9ph"]
Oct 30 22:57:33.802906 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:33.802811    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-qzpg5_b7de093c-a605-4406-a940-e3e59abcb9a0/manager/0.log"
Oct 30 22:57:33.821300 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:33.821257    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v9zbs/perf-node-gather-daemonset-4l9ph" event={"ID":"551da8e0-4504-4c15-910c-d88c9aa223fc","Type":"ContainerStarted","Data":"f7270d0c8926908e9877c7ba13d760690794494e99c03e7f9a98a81118c108cd"}
Oct 30 22:57:33.821518 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:33.821356    4871 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v9zbs/perf-node-gather-daemonset-4l9ph" event={"ID":"551da8e0-4504-4c15-910c-d88c9aa223fc","Type":"ContainerStarted","Data":"e22d10b7e16cffeef032f1c2019e86d49ff1184a30fa41cb539f0030aa1f599d"}
Oct 30 22:57:33.821518 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:33.821381    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-v9zbs/perf-node-gather-daemonset-4l9ph"
Oct 30 22:57:34.144949 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:34.144911    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-qzpg5_b7de093c-a605-4406-a940-e3e59abcb9a0/manager/0.log"
Oct 30 22:57:34.410193 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:34.410091    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-r5c8j_4f069634-b231-4fd0-847e-52cb47ef0163/manager/0.log"
Oct 30 22:57:34.664083 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:34.664000    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-84769bb5bf-j5wbn_888e372b-f344-412f-9f59-6411a4f1177e/manager/0.log"
Oct 30 22:57:35.062546 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:35.062464    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-84769bb5bf-j5wbn_888e372b-f344-412f-9f59-6411a4f1177e/manager/0.log"
Oct 30 22:57:35.317023 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:35.316918    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-pfgpx_5754b4e2-2ccc-43ee-b1d2-e065d15345ca/manager/0.log"
Oct 30 22:57:35.569377 ip-10-0-139-46 kubenswrapper[4871]: E1030 22:57:35.569288    4871 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-h6hzg" containerName="extractor"
Oct 30 22:57:38.938323 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:38.938291    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx_8d401654-8a4c-4a79-b2d3-612765312421/extract/0.log"
Oct 30 22:57:38.949369 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:38.949337    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_e644421527c169d876fdb59235c927986d6bd2881f72d188068392be715p9zx_8d401654-8a4c-4a79-b2d3-612765312421/extract/0.log"
Oct 30 22:57:38.950406 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:38.950383    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/2.log"
Oct 30 22:57:38.957292 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:38.957268    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzntd_2360fddc-52a5-42a5-82ca-a05aff84d217/ovn-acl-logging/0.log"
Oct 30 22:57:38.960832 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:38.960814    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7tf9f_b7002d6d-834c-4bdc-907e-805edd577a40/console-operator/2.log"
Oct 30 22:57:38.967572 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:38.967553    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzntd_2360fddc-52a5-42a5-82ca-a05aff84d217/ovn-acl-logging/0.log"
Oct 30 22:57:39.839100 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:39.839074    4871 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-v9zbs/perf-node-gather-daemonset-4l9ph"
Oct 30 22:57:39.968401 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:39.968339    4871 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-v9zbs/perf-node-gather-daemonset-4l9ph" podStartSLOduration=7.968324647 podStartE2EDuration="7.968324647s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:57:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:57:33.944550223 +0000 UTC m=+4796.122038839" watchObservedRunningTime="2026-10-30 22:57:39.968324647 +0000 UTC m=+4802.145813288"
Oct 30 22:57:40.794734 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:40.794698    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-884576c57-wltlg_65827025-dc76-42d8-be9f-bdfed6b28429/cluster-monitoring-operator/0.log"
Oct 30 22:57:40.881988 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:40.881951    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-qzpg5_b7de093c-a605-4406-a940-e3e59abcb9a0/manager/0.log"
Oct 30 22:57:41.186713 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:41.186677    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-qzpg5_b7de093c-a605-4406-a940-e3e59abcb9a0/manager/0.log"
Oct 30 22:57:42.012677 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:42.012639    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-84769bb5bf-j5wbn_888e372b-f344-412f-9f59-6411a4f1177e/manager/0.log"
Oct 30 22:57:42.313313 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:42.313205    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-84769bb5bf-j5wbn_888e372b-f344-412f-9f59-6411a4f1177e/manager/0.log"
Oct 30 22:57:44.420772 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:44.420744    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5wmsz_70fe39fd-b378-432b-b3bf-4754d342adfb/node-exporter/0.log"
Oct 30 22:57:44.828739 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:44.828654    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5wmsz_70fe39fd-b378-432b-b3bf-4754d342adfb/kube-rbac-proxy/0.log"
Oct 30 22:57:45.236641 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:45.236612    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5wmsz_70fe39fd-b378-432b-b3bf-4754d342adfb/init-textfile/0.log"
Oct 30 22:57:48.307514 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:48.307433    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-575956cc9c6mmf7_3419e5d7-f2ed-43e1-89e0-baccdb485f1b/istio-proxy/0.log"
Oct 30 22:57:48.498266 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:48.498225    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-575956cc9c6mmf7_3419e5d7-f2ed-43e1-89e0-baccdb485f1b/istio-proxy/0.log"
Oct 30 22:57:49.489679 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:49.489633    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-l8t6z_1817be16-1033-45eb-8499-5c2d201bbc2e/discovery/0.log"
Oct 30 22:57:49.687474 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:49.687443    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-l8t6z_1817be16-1033-45eb-8499-5c2d201bbc2e/discovery/0.log"
Oct 30 22:57:50.029147 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:50.029107    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7df854bf67-vjrqn_cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d/istio-proxy/0.log"
Oct 30 22:57:50.223650 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:50.223610    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7df854bf67-vjrqn_cc9ba2a0-4d99-4e16-96e5-0db93a3ec75d/istio-proxy/0.log"
Oct 30 22:57:50.237182 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:50.237149    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-668d64f647-59zws_54b7cb62-6e43-40d4-afbb-2267c98b8c52/istio-proxy/0.log"
Oct 30 22:57:50.434288 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:50.434259    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-668d64f647-59zws_54b7cb62-6e43-40d4-afbb-2267c98b8c52/istio-proxy/0.log"
Oct 30 22:57:50.871580 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:50.871550    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-57f669cfc9-d755h_e0535562-96d7-4fc4-a5d5-ad4cc785080e/router/0.log"
Oct 30 22:57:51.074199 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:51.074166    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-57f669cfc9-d755h_e0535562-96d7-4fc4-a5d5-ad4cc785080e/router/0.log"
Oct 30 22:57:56.061217 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:56.061178    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-575956cc9c6mmf7_3419e5d7-f2ed-43e1-89e0-baccdb485f1b/istio-proxy/0.log"
Oct 30 22:57:56.567001 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:56.566957    4871 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vb42_6d9d43c2-40da-47a8-adb4-d3ba378d77d9/konnectivity-agent/0.log" limit=1024
Oct 30 22:57:56.970619 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:56.970580    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-64d8b4548c-bhhfv_89d9923c-dc8d-4655-be5d-ed3c9c36d7b2/authorino/0.log"
Oct 30 22:57:57.085920 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:57.085884    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-vqwsv_b87c93be-5f5a-45ba-b820-ebd97c1a1c03/manager/0.log"
Oct 30 22:57:57.298121 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:57.297996    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-p8zgz_9c392070-49cb-4c08-8230-19737bd80de4/manager/0.log"
Oct 30 22:57:57.344594 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:57.344561    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-64d8b4548c-bhhfv_89d9923c-dc8d-4655-be5d-ed3c9c36d7b2/authorino/0.log"
Oct 30 22:57:57.467930 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:57.467896    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-vqwsv_b87c93be-5f5a-45ba-b820-ebd97c1a1c03/manager/0.log"
Oct 30 22:57:57.601728 ip-10-0-139-46 kubenswrapper[4871]: I1030 22:57:57.601646    4871 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-l8t6z_1817be16-1033-45eb-8499-5c2d201bbc2e/discovery/0.log"